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SUMMARY 


An investigation of the properties important for the design of stochastic adaptive 
controllers for the higher harmonic control of helicopter vibration is presented. 

Three different model types are considered for the transfer relationship between the 
helicopter higher harmonic control input and the vibration output; 1) Nonlinear, 

2) linear with slow time varying coefficients, and 3) linear with constant 
coefficients. The stochastic controller formulations and solutions are presented 
for a dual, cautious, and deterministic controller for both linear and nonlinear 
transfer models. 

Extensive simulations are performed with the various models and controllers. It 
is shown that even for a linear model the cautious adaptive controller can sometimes 
result in unacceptable vibration control including an apparent controller diver- 
gence. This is found to occur for both constant parameter conditions representative 
of steady flight and time varying parameter conditions representative of maneuvering 
flight conditions. 

A new second order dual controller is developed which is shown to modify the 
cautious adaptive controller by adding numerator and denominator correction terms 
to the cautious control algorithm. The new dual Controller is simulated on a 
simple single-control vibration example and is found to achieve excellent vibration 
reduction and significantly improves upon the cautious controller. 

Nonlinear, time varying coefficient and constant coefficient systems are each 
found to exhibit distinctive characteristics using the adaptive controllers. Simula- 
tion and analysis are presented in an attempt to further understand the closed loop 
behavior. 
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1. INTRODUCTION 


The use of adaptive controllers for higher harmonic control (HHC) to eliminate 
helicopter vibration has been investigated in numerous analytical studies [1, 

2, 3, 4, 5], wind tunnel tests [1* 7, 8 ] and flight tests [9, 10]. The 
effectiveness of higher harmonic control has been experimentally verified and found 
to work exceptionally well in most cases. However, specific cases have been found 
where the use of an adaptive controller either failed to converge to the true mini- 
mizing solution, failed to simultaneously reduce several vibration components, or 
have shown a tendency for divergence. This has been reported in analytic studies 
[17, 22], wind tunnel tests [1, 7] and flight tests [9, 10]. 

The adaptive closed loop controllers can be considered to have attained the proof 
of concept status. However, there is a lack of understanding of the intrinsic con- 
troller properties. Specifically, there does not exist at this time a stability 
analysis for the closed loop system. Cases exist where the controller fails to 
fully minimize vibrations and this is not fully understood. The problem is further 
complicated by the various model descriptions of the helicopter HHC input to vibra- 
tion output. The helicopter vibration model which is assumed to be in quasi-steady 
state can exhibit nonlinear, linear time varying, or linear constant coefficient 
behavior. In addition, the vibration model is in a sense stochastic since the trans- 
fer matrix varies in an unknown manner to a certain degree with flight condition and 
since the measurements are contaminated by random noise. 

Because of the diversity of helicopter vibration model types and the stochastic 
nature of the environment, it is difficult to generalize the properties of the closed 
loop system. A solution which shows excellent convergence during one flight condition 
may be unacceptable during another condition. The difficulty is further complicated by 
the fact that the adaptive control algorithm is itself nonlinear and its properties 
cannot be analyzed by linear methods. 

There are several variations for the implementation of the adaptive controllers 
used for helicopter vibration minimization. These have been referred to as local or 
global models. They can be implemented with and without stochastic properties (such 
as cautious and probing) and can be designed to minimize various optimization criteria 
to include specific vibration states, HHC control inputs and rate of change of HHC 
input. Extensive simulations have been done using the various controllers as 
reported in detail in [1-8 ]. Although the various controllers can 
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be implemented differently they all contain an adaptive parameter identification 
algorithm followed by a controller which utilizes the identified parameters. 

It is of importance to further understand the closed loop adaptive system 
properties. The adaptive control implementation can result in a divergence. In 
addition, depending upon the nature of the random disturbance the controller can 
fail to minimize vibrations under certain conditions . This behavior would 
certai nl y be unacceptable for use on production helicopters. 

This report presents results for the three helicopter vibration model descrip- 
tions; 1) nonlinear, 2) linear time varying and 3) linear constant coefficient. 
Simulations are presented which show specific conditions when the closed loop 
adaptive controller exhibits divergence, or fails to fully minimize vibrations. An 
investigation of the properties of the closed loop controllers is presented and a new 
dual controller is developed. 

A stochastic adaptive control formulation is presented for nonlinear and linear 
model descriptions. A new second order dual control solution is developed in an 
attempt to improve upon the non-dual controller (i.e. cautious or deterministic). 

The new dual controller is such as to modify the cautious adaptive controller by 
adding numerator and denominator correction terms to the cautious control algorithm. 
Conditions are simulated where the cautious controller fails to minimize vibration 
and the new dual controller is found to achieve excellent vibration reduction and 
convergence. Although, the new dual controller is developed for a multi- input/multi- 
output vibration problem, the simulations are presented for a scalar example. 

The theoretical background and HHC problem formulation is presented in section 
2. Stochastic control solutions are presented for both the linear and nonlinear 
problem descriptions in section 3. The simulation models used are discussed in 
section 4 and the details of the various closed loop simulation results are paesented 
in section 5, The derivation of the new second order dual controller is presented 
in the appendicies. 
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2. THEOEETICAL BACKGROUND 

Helicopter vibration can be reduced by active blade control using controllers 
employing stochastic control theory • Various control policies are available and 
research is on going to extend these ideas for further improvement in vibration 
reduction. A summary of the vibration problem and various control approaches is 
provided here. 


2.1. Higher Harmonic Control Problem Formulation 


The linear multivariable model representing the higher harmonic control 
(HHC) input to output vibration is 

x(k+l) = c(k) + B(k) u(k) (1) 

where c(k) is an unknown vector and B(k) is a matrix of unknown parameters. The 
unknown elements of c(k) and B(k) are denoted as 0(k) with covariance matrix P(k). 
These unknown parameters are time varying and their variations are modelled as 
6(k+l) = A 6(k) + v(k) (2) 

with 


E{v(k) } = 0 and E{v(k) v’(j)} = V 5... (3) 

In equilibrium flight the steady outputs (vibrations) at zero control is c(k) . 

This is a static model where it is required to find the control u(k) which reduces 
the uncontrolled vibration c(k) . 

The measurement equation is 

y(k) = x(k) 4- w(k) (4) 

where 


E{w(k) } = 0 ; E{w(k)w'(j)} = W 6 

E{w(k)v' (j) } = 0 


(5) 


and x(k) , y(k) being n dimensional vectors. The general control criterion to be 
minimized is the expected value of the cost from step 0 to N 


N 

J(0) = E{C(0) } = E{ E x'(k) Qx(k) + u'(k-l) Ru(k-1) } (6) 

k=l 

The nonlinear HHC problem formulation is given next. The nonlinear relation- 
ship between HHC input and vibration output can be approximated by the Volterra 
harmonic series developed in {17] . The input-output nonlinear relationship 
including up to third order terms is. 
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x(k+l) = cCk) + B(k)uCk) + i u'(k) S(k)u(k) + u ' (k) T Ck) u 2 (k) £7) 

This relationship represents both the steady and the varying flight conditions. 
During varying flight conditions the coef ficients c(k) , B(.k). , S(k) and TCk) vary 
according to Equations 2 and 3. The measurement equation and the control criterion 
are the same as before. 

The solutions to these problems are given in Section 3. 

2,2 Stochastic Control Theory Background 


In most real world systems there are inherent uncertainties that prevent 
the use of deterministic control theory. These uncertainties, either in the 
system itself or in the measurements made on the system, can be appropriately 
modelled as stochastic processes. An open-loop controller does not require 
any measurement of the system. It is a function of the initial state and time. The 
feedback controllers utilize real-time observations. 

In a stochastic environment the control has a dual effect [11, 12]: it 

affects the system’s state as well as the uncertainty in the estimated parameters. 
This dual effect can be utilized for designing good controllers for the vibration 
reduction in a helicopter. It enhances estimation of the unknown parameters but 
keeps the output at a recommended level. 

Let us consider a stochastic system with unknown parameters 9 where the 
state of the system at time k, x(k) evolves according to the equation 

x(k+l) = A(0) x(k) + B(0) u(k) + v(k) £8) 

where u(k) is the control applied at time k and v(k) is the process noise. 
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( 9 ) 


The measurement is given by 

y(k) - H x(k) + w(k) 
where w(k) is the measurement noise. 

We are interested in a fixed end-time problem and the performance index to be 
minimized is 


min J = min E{ C(0, x N , U N 1 ) } (10) 

h ®’ 1 u 11 - 1 

where 


N-l 

C(0, X N , U N_1 ) = x'CN) Qx(N) + Z x'(k) Q(k) x(k) + u'(k)R(k) u(k) 

k=0 


(ID 


l ?- 1 £ 


N-l 

{ u(i) } 

i=0 


( 12 ) 




A “ 

- { x(i) } 

i=0 


(13). 


In an equivalent deterministic situation, (known 9 and no noises) the parameters 
and the state variables are known exactly and we are interested in 

min C(0, X N , u* 1-1 ) ( 14 ) 

Z " 1 

The deterministic controller is 

U D (k> = L[k, 9] x(k) (15) 

If the parameters were known, but the noise existed, then the stochastic con- 
troller for this problem (LQ) has the Certainty Equivalence (CE) property [12,20,21] 

U S (k) = L[k, 9] x(k|k) (16) 

For a plant with, unknown parameters the stochastic controllers are adpative in 
nature and the parameter 0 is estimated in real-time. 
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A stochastic controller that ignores the uncertainties in the parameter 
estimates is the Heuristic Certainty Equivalence controller 

uHCE(k) _ ^fk^ 0 (k) ] x(k jk) (17) 

In the Self Tuning Regulator (STURE) perfect state observations are available 
and unknown parameters are estimated in real time. The controller is 

u STUBE (k) = L[k> Q (k)] x(k) (18) 

A cautious controller belongs to the feedback, class 1 12,20,21] and makes use 
of the uncertainties in the parameter estimates. It is given by 

u CAUTI0US (k) a L[k> Q( k) s P ( k )] x(k|k) (19) 

assuming 

P(j). = P(k) V j > k C2Q1 

All the above mentioned controllers exhibit dual effect in a stochastic environ- 
ment (non-neutral) but none of the above uses the dual effect in its design. The 
dual controller incorporates this and thus belongs to the closed-loop class and 
anticipates future learning. It takes into account the functional dependence of 
future covariances on the current control u(k) . It is given by 

u CL (k) = arg min J [k, x(kjk) , 0(k) , P(j|j), j >_ k] (21) 
u(k) 

This often leads to a heavy computation load [ 13 } 2 . 4 , 15, 16], O ' 117 idea 
is to get a control of the form 

u(k) » L[k, 6 (k) , P(k) , , j> k] x(k|k) (22). 

which is simpler to handle and analyse. 
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3, STOCHASTIC CONTROL SOLUTION 


This section presents the control solutions for the deterministic, the 
cautious, the first order dual and the second order dual controller based upon 
a linear transfer matrix vibration description. In addition, the nonlinear 
stochastic formulation and solution is also presented. These controllers are 
evaluated by simulation in subsequent sections. 

The general linear and the nonlinear problems are formulated in previous 
sections and the various stochastic control policies are discussed in Section 2. 
The different solutions to the linear and the nonlinear problems are given here. 
Three objective functions for the design of the controllers are: 


Deterministic : 

J DET = (k)Qx(k) + u' (k-l)Ru(k-l) + Au* (k-1) 

RAu(k-l)} (23) 

Cautious : 

J CAUT = S ^ J DET ^ 

(24) 


N 


Dual: 

■w - E{ J 1 W 

(25) 


A dual solution is presented for N - 2 based upon an approximate 
linearization. 

Solution to the Linear HHC Problem 

For the linear HHC problem with R = 0 we have the following control solutions 
derived in [3,4] and Appendix A: 

u D (0) = -[B'OOQ B(k) + R] -1 [B f (k) Q cCk) ] ; (26) 

u T( 0 ) - - [B 1 (0) QB(0) + Z q^PgCO) + R] A [B’(0)Qc(0) + Z P^CO) ] ; (2 7) 

'£.““1 ^ r“l 

u* Cl) C0) = -[B*(0)QB(0) + £ q. p£(0) + R]" 1 [B T (0) Qc(0) + 

1=1 Z B 

n l 

X Cq £ P B C (0) + f £ ) ] ( 28 ) 

where the vector f^ is 
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(29) 


f l ~ 


n n 
Z E 
i=l j=l 


1 3J (1) 

2 


9P £ .(1) 

i.J 


<i (1) 

3u(0) 


The superscript (1) in indicates the dual solution is derived based 

upon a first order Taylor series expansion as developed in [3,4], The partials are 

A 

evaluated at the nominal cautious control u (0.) , the parameter 6 CO) and the nominal 
covariance P(l) . The new dual control (Appendix A) is 


u u; (0) 


-[B'CO) QB(0) + E (q„ pf(0) + F„) + R] 1 [B(0) Qc(0) 
1=1 ^ B ‘ c 


X 


+ 2 Cq £ PgcCQ) + f £ )] 


where the matrix F^ and the vector are 


(30) 


f £ = 


n n 


1*1 j=l 


/ * 

/ 3J (1) 1 3 

. 3J*(1) \ 

3 

3P £ .(1) 
A»J_ 

\ 9p5 . (1) 2 36^(1) 

1 9 3 1 

39^(1) / 

* 3u(0) 

3u(0) 


(31) 

u I (0),6(0),P(l) 


to = 


n n 
E E 
1=1 j=l 


l/ 9J*(1) 

1 3 

; 3J*ci )\/ 

2 ' 8P i , j (1) 

? A £ 
307 ( 1 ) 

3 ^( 1 ) )\ 


3u(0) 


3u(0) 3u(0) 


^ u (0) 


u (0), 0(0) , BCD 


(32) 


where t - 1, 2, . ... n row of B and P . . is the i,jth element of the covariance 
matrix P. 

( 2 ) 

The superscript (2) in \i' CO) indicates the dual solution is derived based 

DUAL 

upon a second order Taylor Series expansion as developed in Appendix A. 

3.2 Solution to the Nonlinear HEC Problem 


The deterministic nonlinear problem is to determine the HHC which minimizes 
the quadratic criterion of Eq. (23) subject to the nonlinear model of Eq. (7) . The 
stochastic one-step problem uses the criterion of Eq. (24) and the stochastic 
multi-step problem uses the criterion of Eq. (25). 
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The nonlinear equation shown in Eq. (7) is a global nonlinear model (using 
the definition of [3,4]). Taking the difference between two successive time points 
results in a local nonlinear model (this was formally done in [22] for the linear 
case). Ignoring nonlinear terms in Eq. (7) results in the global linear model . 
Ignoring nonlinear terms in the local nonlinear model results in the local linear 
mo del . Thus, four possible representations can be derived from Eq, (7) ; 

1) global nonlinear, 2) local nonlinear, 3) 'global linear, and 4) local linear. 

The development to follow will only consider the global nonlinear model of 
Eq. (7) and deterministic criterion of Eq. (23). Although it is possible to treat 
the stochastic criteria and local nonlinear models, this will not be included here. 
A subsequent section discusses a nonlinear stochastic solution. 

3.3 Open Loop and Adaptive Higher Harmonic Control 

The minimization of the deterministic cost function Eq. (23), subject to the 
global nonlinear model Eq. (7) is a problem in nonlinear programming. This is 
equivalent to open-loop (or off-line) optimization. There are many optimization 
techniques available for optimization of a nonlinear function of many variables 
and [24] provides a detailed examination of such techniques . Two optimization 
techniques which require explicit calculation of the gradient of the cost function 
are the gradient method and Newton’s method [24], Two optimization methods which 
do not require explicit gradient calculation are Powell's method and Rosenbrock's 
method [24 ] . 

Newton's method is presented here as an example of an open loop optimization 
method. It will also be shown that under suitable approximations this method 
leads to the open loop global adaptive controller of [1, 3, 4, 22], 

Newton's method [24] for minimization of the cost function J of Eq. (23) 
subject to the nonlinear model (Eq. 7) is 

u 1 = u 1 " 1 - atV 2 !]"^ VJ I _ 1 (33) 

where I represents the iteration number, a is a scalar step length parameter 
(0 _< a _< 1) , V J represents the gradient of J with respect to and Y^J 

represents the second gradient (Hessian) . 

The gradient VJ and Hessian J are computed at 1-1 from Eq. (23) which 
yields 
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Qx + Ru+RAu 


(34) 


_2 3 ^ x ' ^ X ' 

= ^ 9T<! * + *r 


Q % + R+ R 


where the vector x represents the vector of vibration components of Eq. (7). 

In order to use the open loop Newton control of Eq. (33) , the gradients 
3x f „ / 9x T \ 


~ q u ~~ and ^ Q J x in Eq. (34) and (35) must be explicitly determined. 

A second algorithm is developed using an approximation to V^J. A simplification 
to Eq. (35) results if the second gradient terms are ignored. Thus an approximation 
form for vb is 


v J ' &r « + E + *• C36) 

Thus, an approximate Newton algorithm using Eq. (33), Eq. (34) and Eq. (36) is 

I i_i 3x T -1 

u = u - a -x-— Q -gjj- + R + R 


Q x + R u I_1 + R Au 1 1 (37) 

If we assume a linear transfer matrix for Eq. (7) (i.e. S = T = 0, in Eq. (7) 

then 

x = c + Bu (38) 

where B represents the linear transfer matrix. 

Substitution of Eq. (38) into Eq. (37) and assuming R = 0 yields 

u 1 - u I_1 - a[B f QB + R] _1 [B'Q(c + Bu 1-1 ) + R u 1-1 ] (39) 

If we assume a =1, Eq. (39) reduces to 

U 1 = - [B’$B + R]' 1 lB , Qc] (40) 

which has the identical form as the global linear adaptive controller with on- 
line identification of B and c, For on-line identification the iteration number 
I is replaced with time step number i. 
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Thus, the global linear adaptive controller of [1,3,4] can be viewed as a on- 
line version of a Newton method of optimization using an approximation to the 
Hessian V 2 ! and assuming a linear transfer matrix. It is also possible to start 
with the local nonlinear model formulation and under suitable approximation 
develop a on-line Newton type algorithm which uses the local linear transfer 
matrix approximation. 

The above example development of a Newton type adaptive controller provides 
insight into convergence behavior since off-line Newton algorithms have been 
extensively researched. In [25] (pp. 392-400) a further comparison of two 
iterative methods for solving the roots of a nonlinear equation is discussed. 

The first method called the Method of Chords is analogous to the global linear 
model assumption. The second method called the Method of Tangents is analogous 
to the local linear model assumption. Convergence behavior of the adaptive 
controllers is further understood with comparison to these two iterative methods. 

The global linear cautious controller of [1, 7,22,23] is found to exhibit 
good convergence to a local minimum solution for a nonlinear transfer relationship. 

The caution property is a result of including the parameter covariance matrix 
in the control solution [1], The improved convergence with the caution property 
is similar to the off-line nonlinear optimization method referred to as the 
Marquardt method in [26] . The Marquardt method [26] is similar to Newtons method 
where the control is iterated using 

u 1 = u 1 " 1 - [V 2 J + a I]" 1 VJ (41) 

where is a scalar parameter multiplied by the identity matrix I. Eq. (41) 
is identical to the Newton algorithm of Eq. (33) with cf=1 if o^=0 in Eq. (41). 

The term a^I in Eq. (41) is used to improve convergence. When the control 
is far removed from the minimum solution is set to a very large number then 
the matrix inverse in Eq. (41) is a small value approximately equal to I/a^. Thus 
the algorithm is similar to the gradient method 

u 1 - u 1-1 - l/a 1 VJ (42) 

The gradient method is known to converge better than the^ Newton's method when far 
removed from the minimum solution (i.e. when the control is in a very non- 
quadratic region) . As the minimum solution is approached is then set to a small 
number approaching zero and Eq. (41) behaves like Newton's method of Eq. (31), 
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Newton's method has superior convergence to the gradient method in the vicinity 
of the minimum solution (it has quadratic convergence) . 

Eq. (41) is seen to be very similar to the global linear cautious controller 
where the covariance matrix P^_^ is used to replace a^I in Eq. (41) . The co- 
variance matrix is set to a large number initially and decreases as the parameters 
are more accurately identified. Thus the cautious controller can be viewed as 
behaving initially like a gradient algorithm and then like a Newton-type algorithm 
as the minimum solution is approached. This explains the excellent convergence 
behavior observed in most cases with the cautious controller. 

Adaptive controllers require on-line estimation of the parameters of the 
model of HHC input to vibration output. If the nonlinear model (Eq. (7)) is used 
then the parameters are contained in c , B, s, and T. A linear Kalman filter 
can still be used since the parameters enter linearly. 

This section presents a brief treatment of open-loop and adaptive HHC for 
the nonlinear problem. A large number of both open-loop and closed-loop adaptive 
algorithms can be developed using the formulation presented. Four categories of 
plant model are possible: 



Local 

Linear 

(!) Nonlinear Programming Methods 

(2) Off-line or on-line based upon Nonlinear Programming Method Chosen 

As is apparent from these four categories a large number of combinations exist 
for both open-loop and adaptive algorithms. Although both the local and global 
linear adaptive controllers successfully converge to local minimum values of the 
cost function for a nonlinear vibration model (this is shown in [22, 23]) two specific 
problems remain due to nonlinearity; 1) multiple minima solutions exist, some of 
which do not yield sufficiently low vibration levels and 2) adaptive HHC 
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algorithm divergence. The nonlinear formulation presented in this section provides 
the mathematical framework to further analyze these two specific problems. 
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3.4 Nonlinear HHC Theory and the Dual Solution 
The dual theories ’developed earlier may be extended in a straightforward 
manner to a nonlinear model. It is assumed that for at least some flight conditions 
the mathematical model representing the helicopter vibration is nonlinear. A non- 
linear model has been proposed in [17, 22]/ This provides the motivation to extend 
existing dual control solutions [13,14,18] to handle a nonlinear model. For 
simplicity, here a scalar nonlinear model is selected and the algorithmic steps 
are summarized. There is a distinctive difference in the complexity of this algorithm 
This will be indicated in the proper context but the concepts are similar to that of 
the linear case. 

Let us consider a plant 

x(k+l) = d + eu(k) + fu^(k) (43) 

where d 5 e 5 f are unknown, but time invariant scalars. These unknown elements d , e, 
f are denoted as 0(k) with covariance matrix P(k). Since the parameter set 
{ C d, e, f ) £ 0(k) } is time invariant we represent it as 

0(k+l) = 0(k) (44) 

The measurement of x(k) is according to 

y(k) = x(k) + w(k) (45) 


where 


E(w(k)) = 0 and E{w(k)w(j) } - W 6. 


kj 


(46) 


The control criterion to be minimized is the expected value of the cost from 
step 0 to N 


J(P) = e{ CCO) } = E{ E q x 2 Ck) + r u 2 (k-l) } CA7) 

k=l 

where N = 2 for the two-step ahead criterion. 

The minimization of (47) with respect to u(0) and u(l) subject to (43)~(.46) 
is obtained from the" Stochastic Dynamic Programming equation [19,20]. 

J (k) = min E{c(k) + J (k+1) |/} k = N-l, ... , 1, 0 (48) 

u(k) 

where J (k) is the "cost-to-go" from k to N and I is the cumulated information 
at time k when the control uCk) is to be determined. For N = 1, Eq. (48) is 

J*(0) = min E{q x 2 (l) + r u 2 C0) + J*(l) |I 0 } (49) 

u(0) 
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where J Cl) is the optimal cost at the last step and is obtained by minimization 
of J(N-l) for N = 2. 

Thus, 

J (1) = min E{q x 2 (2) + r u 2 (l) |lh 
u(l) 

= min E{q ( d + eu(l) + fu 2 (l))^4- r u 2 (l) jl"^} 
u(l) 

= min E{q(d^4e^u^(l) + f 2 u^(l) + 2deu(l) 4- 2dfu 2 (l) 

uCl) 

+ 2ef u 3 (l) + r u 2 (l) ll 1 } (50) 

The control u(l) can be computed only by a search procedure on this stochastic 
& 

surface J (1) . Here the method differs from that for the corresponding linear 
plant case and no explicit elimination of uCl) in terms of the plant parameters 
is possible. 

Combining (49) and (50) we get, 

J (0) = min E{qx 2 (l) + ru 2 (0) 4- J*(l)jl°} 
uCO) 

= min E{q(d 2 4e 2 u 2 (0) + f 2 u^(0) + 2deu(0) 4- 2ef u 3 (0) + 2dfu 2 (0)) 4- ru 2 (0) 

u(0) 

4- min E{q(d 2 4e 2 u 2 (l) + f 2 u^(l) 4- 2deu(l) 4- 2efu 3 (l) + 2dfu 2 (l)) 
uCl) 

+ ru 2 (l) ll 1 }]! 0 } (51) 

From (50) and (51) it is evident that 

J*(l) = f 1 (6(l) , P(l), u(l)) (52) 

from which 

u*(l) = f 2 ( 0(1) , PCD) (53) 

Combining (52) and (53) we get, 

J*(l) = ^[0(1), P(l), f 2 ( 0(1) , p(l))] (54) 

where f -(.,., .) is an explicit function but f is not one, because u (1) can 

i /s. A 

be obtained only by a search if 0(1) , P(l) are known. Thus unlike the linear case, 
uCO) also has to be obtained by a search and no explicit formula can be derived for 
it. However, an algorithm based on the dual control ideas applied to a linear 
plant [ 13,14,18] can be summarized as follows: 
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(1) use a nominal cautious control u(0) obtained by a search method 

(2) get a nominal P(l) 

/s. 

03) obtain another control u(l) = f 0 (900), P(l)) again by search and 

A 4 

(4) linearize J(l) about 0(0), P(l) , u(0) as 


J*U) = J*[ 9(0) , P(l) ] 


3J*(1) 

3P(1) 


u(l)=u(l) 


• (P(l) - P(l)) 
u(l) , 0(0) ,P(1) 


. 3J*(1) 

+ X 1 11 — - 

30(1) 


• (9(1) - 0(0)) 

A 

0(0) ,P(1) ,u(l) 


+ i (0(1) - 0(0)) ' • (0(1) - 0(0)) (55) 

30 C1) 8(0) ,p(D ,5(1) 

The covariance P(l) is influenced by u(0) according to the linear Kalman 
Filter equation. Thus we may linearize P(l) about the control u(0) according to 


PCD = P(l) + 


3P(1) 

3u(0) 


(u(0) - u(0) ) 


u(0),P(0) 


+ |(u(0) 


u(0) ) ' 


3 2 P(1) 

0u 2 CO) 


* CuCQ) 
5(0) ,P(01 


u(0).) 


(56) 


Using (51), (55), (56) we get u(0) b.y a final search, method.. Conceptually, 
the extension to the nonlinear plant is not difficult but the complexity is increased 
many fold. 
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4. SIMULATION MODELS 

This section discusses the three distinct mathematical models used to represent 
-the relationship between the helicopter higher harmonic control inputs and its 
vibration outputs. The model descriptions are: 

1) a nonlinear transfer matrix model 

2) a linear time varying transfer matrix model, and 

3) a scalar model representing a single HHC input to a single vibration 
output . 

4.1 Nonlinear Model 

A third order polynomial model has been derived in [23] . The identified 
longitudinal hub force polynomial model at 120 knots flight condition is shown in 
Figures 1-3. This longitudinal hub .vibration model as a function of the two 
control inputs 9g c and Q^ s is described by the polynomial equations, 

Xl (k+1) = 108.9 - 74.84 Ul (k) - 51.04u 2 (k) - 6.515u 2 (k) - 2.825u 2 (k) 

- 1.04u 1 (k)u 2 (k) + 7. 77u 2 (k) + 22.27u 2 (k)u 2 (k) C57) 

+ 5.92u 1 (k)u 2 Ck) + 4.17u 2 (k) 

x 2 (k+l) = -89.74 + 53.31u 1 (k) - 82,56u 2 (k) - 7.42u 2 (k) + 7.34u 2 (k) + 1.91 Ul Ck)u 2 (k) 
+ 5.63u 2 (k) + 26.24u 2 (k)u 2 (k) 7.09u^(k) u 2 (k) + 6.54u 2 (k) (.58) 

where 

x^ = longitudinal hub force (cosine component of vibration), LBS. 
x 2 = longitudinal hub force (sine component of vibration) , LBS, 
u^ = cosine component input, 0^ , deg, 
u 2 = sine component input, 9g g ' , deg. 

and measurements according to 

y 1 (k) - x 1 (k) + w 1 (k) 

y 2 (k) = x 2 (k) + w 2 (k) 

where it is assumed that 

E{w(k) w'Cj)} = W 6 = diag (10. 89 2 , 8. 97 2 ) (fill 

K-3 

Figures 1 and 2 show the longitudinal cosine vibrat ion x^ and the sine 

vibration x 2 respectively vs. the control inputs 9^ and 9^. The nonlinearity is 

clearly shown for 1 0„ | 1° arid | 0„ [ _> 1° . The total longitudinal cost function 

22 oc Js 

x^ + x 2 is shown in Figure 3. 


(59) 

(60) 
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4.2 Linear Time - Varying Multivariable Model 
The parameters of the helicopter vibration model can be rapidly 
varying during an acceleration or deceleration maneuver. A simulation of 
this situation is done using a linear transfer matrix model with varying 
parameters . The current algorithms are evaluated on this model. The HHC 
problem formulation for time varying parameters assumes that the transfer matrix 
elements B(k) and the vibration components without control c(k) are modelled as a 
random walk 

eCk+1) = 0(k) + v(k) (62) 

where 

{c, Be6} (63) 

The plant process noise v(k) is zero mean white gaussian of 
covariance V . The process noise covariance V is assumed a value of 30% of the 
initial parameter values so that they simulate a. rapid maneuver. 

The linear vibration model used is 

x(k+l) = c(k) + B. (k) u(k) (64) 

where x(k+l) is the vibration when HHC u(k) is applied. This vibration is measured 

y(k) = x(k) + w(k) (65) 

The measurement noise w(k) has a standard deviation of 10% of the uncontrolled 
vibration at time t=0. The random walk model of (62), (63) can describe the 
helicopter in any number of transient flight conditions. 

(1) Helicopter transition from hover to forward speed (or from forward 
speed to hover) 

(2) Helicopter in a windy environment 

(3) Maneuvering flight conditions, and 

(4) Nonlinearity (assuming nonlinear effects can be modelled as 
linearized time-varying parameters) . 

The simulation model described by (62)-(65) does not describe just one 
helicopter model. Monte-Carlo simulations with various noise levels on the model 
of (62) -(65) can describe the helicopter in several different transient modes or 
perhaps even different helicopters as well in different flight conditions. Thus 
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the model is quite general and can be used to evaluate the existing algorithms under 
different conditions. The simulation is started with the initial linear plant as 


x^k+1) = 108.9 - 74. 84u^(k) 

- 51.04u 2 (k) 

(66) 

x 2 (k+l) = -89.74 + 53.31u 1 (k) 

- 82.56u 2 (k) 

(67) 


and the measurements are according to (59) , (60) . 

4.3. Scalar Model 


A scalar model is used to evaluate the new dual solution based upon 
the second order Taylor series expansion and is given by 


x(k+l) = c + b(k)u(k) 

(.68) 

b(k+l) = a b(k) + v(k) 

(69) 

and the measurement is according to 


y(k) = x(k) + w(k) 

(70) 

with 


E(v(k)) = 0 , E(v(k)v'(j)) = V 6 kj 

E(w(k)) = 0 , E(w(k)w'(j)) = W 6^ 

E{w(k)v'(j) } = 0 

(71) 


For the time varying case, the parameters used are 

b(0) = .05 , P b (0) =1.0 , V = .1 , c = 1.0 

W = .01 and W = .1 , a = .9 

For the constant case, the parameters used are 

b(0) = .05 , P b (0) = 1.0 , V = 0 , c = 1.0 

W = .01 and W = .1 , a = 1.0 
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5. SIMULATION RESULTS 


This section presents the results of the cautious and the dual adaptive 
controllers presented earlier in Sections 2 and 3 using the simulation models 
presented in Section 4. , 


5-1 Simulation Model for the Time Varying 
Multi-variable Linear Model 

These two controllers are based on the linear time varying model discussed 
earlier in Section 4, The process noise covariance of the parameters is selected 
so that it represents a rapidly varying flight condition. The process noise standard 
deviation is selected as 1Q%, 30%, and 50% of the initial parameter values. These 
three cases are referred to as the 10% noise case, 30% noise case and 50% noise 
case. Each simulation run is performed for 40 time steps for 100 Monte-Carlo runs. 

Fig. 4 shows a time history of the cost function using the cautious control for 
these three noise levels. The 10% noise case (circle in Figure 4) shows excellent 
vibration reduction. The vibration reduction is poorer with 30% noise (triangle 
in Figure 4) and the 50% noise ( + symbol) . The 30% noise case shows poor vibration 

reduction for 0 < k < 16 and 32 < k < 40 and good reduction for 16 < k < 32, k being 
the time step. The 30% noise case is selected for more extensive evaluation. Figures 

5 and 6 show the cosine and sine longitudinal components respectively of the amplitudes 

squared for the 30% noise. The Kalman filter of Figures 4-6 is initialized with 
parameters taken from a random number generator with a standard deviation equal 
to the magnitude of the true parameter values. This represents an acceleration run 
of a helicopter where the parameters are poorly estimated at hover. A second case 
is also run in which the parameter estimates were initialized as true values with 
the standard deviation equal to 10% of the true parameters. This represents a 
transient deceleration run after the parameters have been accurately identified 
in steady flight conditions. Figures 4 through 6 provide the motivation for 
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studying in depth the effect of the cautious and the first order dual controller 
[13] on the time varying parameter model. A summary of the results for two cases 
of initialization is discussed next. A detailed discussion of the second case 
will be discussed later on. 

5.2 Controller Performance for the first case 
(Kalman Filter Initialization with large Initial Covariance) 

Figures 4-6 show that the cautious controller is not able to reduce the 

vibrations for the regions 7 < k < 16 and 32 < k < 40. Referring to Figure 4 

(30% noise case) three regions are defined as follows: 

Region 1: k * 7 to k = 16 

Region 2: k = 16 to k = 32 

Region 3: k = 32 to k = 40 

The cautious controller demonstrates good performance in Region 2, but poor 
results in Regions 1 and 3. The dual controller based upon the first order 
linearization [13] is used next and a performance comparison is plotted in Figures 
7-9 for the same Monte Carlo simulations. The design parameter 3 is set to 1 and 2 
[13] . For 3=2 the results are worse than that for the cautious controller and for 
3=1 they are the same as that for the cautious case. Here it is to be noted that 
Regions 1 and 3 are uncontrollable i.e. , a control which neutralizes the vibrations 
does not exist within the limits of + 2°. However Region 2 is controllable but since 
the cautious controller performs so well in this region, the dual controller can offer 
no further improvement. Figures 10-15 show the true parameter variation, the 
estimated parameters and the associated + one standard deviation band from the 
Kalman filter. Figures 11 and 12 show poor identification of the parameters 02 
and 0^ in Region 2 but reasonably good control performance. This is because the 
true and estimated parameters for ©2 and 9^ are far from the zero i.e,, still in 
a controllable mode. A different feature will be demonstrated in the next case. 

5.3 Controller Performance for the second case 
(Kalman Filter Initialization with True Initial Parameters 
and very small Initial Covariance) 

For this run the Kalman filter is initialized with the true parameter values 
and the initial parameter standard deviation is 10% of the magnitude of the initial 
parameters. This simulation differs from the previous one only in the Kalman 
filter initialization. 

Figures 16-20 compare the cost using the cautious control (solid line) with 
the cost using no control (dashed line). The three regions are clearly marked in 
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the figures and the controller demonstrates no improvement over no control in Regions 
1 and 3. From examining the determinant. Region 2 is controllable but Regions 1 and 
3 are not. Thus Region 2 can be improved upon by a dual controller. Figures 17 and 
18 show the cautious controls u^ and • The controls sometimes go to zero and thus 
turnoff occurs. Figures 19 and 20 show the contribution from the individual states. 
Figures 21-26 show the true, estimated parameters and their associated variances. 

These figures indicate the cause of the poor performance of the cautious controller. 

The improvement in Region 2 by a dual controller will be discussed in detail in a 
later sub-section. Figures 27 and 28 show the variation of the determinant of the 
transfer matrix (measure of controllability) and that of the ratio of its eigenvalues 
(degree of controllability) . 

5.4 Controllability Condition for the PI an t 

For a plant with known parameters the controllability condition requires that 
the transfer matrix composed of the parameters 6^, 0^, 0^, 0^ has an inverse, i.e., 

9 2 9 6 ' 9 3 9 5 * 0 

This condition may be violated due to various reasons: 

1) all or some of the parameters individually are simultaneously zero 

2) the transfer matrix consists of linearly dependent rows or columns. 

When such a situation occurs it is impossible to control the vibrations with only a 
limited amount of control. For a plant with unknown parameters, the matrix consisting 

>\ y\ 

of the estimated parameters © 2 , 0g, 9^, 0g also has a role to play. Sometimes, as 
in Region 2 of Figure 16, the parameters are poorly estimated as near zero with 
the wrong sign. In such a case, turn off may occur and the vibration may not at 
all be reduced even though the true transfer matrix is controllable. Probing helps 
in such instances and this has been observed using the dual controller. Detailed 
analysis of these phenomena are discussed next. 

5.5 Detailed Discussion of the controller performances for 
the second case of initialization; Scope of Probing; 

Uncontrollable regions 

Here the Kalman filter is initialized with the true parameter values and the 
initial parameter standard deviation is 10% of the magnitude of the initial parameters. 
The simulation is carried out in a Monte Carlo fashion for 100 runs. Both the 
cautious and the dual controllers are used. Each of these 100 runs is made for 40 
time steps. The results of the average cost per run are tabulated in Table 1, 
using no control, the cautious controller and the dual controller (3 - 1) [13]. 

The average cost over all runs is given at the bottom of the table. Applying no 
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control, the average cost is 50,531. Applying the one step ahead cautious control, 
the average cost is 18,051. This is a reduction of 64.3%. The dual controller 
yields 17,141, a reduction of 66.1%. These figures do not give us any insight into 
the conditions of the plant when probing is useful. Each run requires individual 
analysis. Comparative plots of the performance of no control, cautious control, 
and the dual control for the first 20 runs of this Monte Carlo study are given in 
Figures 29-48. The symbols in these figures are circle- no control, triangle- 
cautious control, and plus symbol- dual control (8 = 1) . These comparative plots 
show clearly that there exist some situations when the dual does better than the 
cautious and when both the dual and the cautious perform well. Four runs are 
selected from Table 1 and are discussed in detail. The runs and the % cost 
reductions are shown Below. 


COST REDUCTIONS FOR 4 SELECTED RUNS 
FROM 100 MONTE- CARLO RUNS 


Run N o . 

No Control 
Cost 

Caut ious Control 

Dual 

Control 

Cost 

% Reduction 

Cost 

% Reduction 

i 

28583 

20215 

29.2 

15764 

44.8 

2 

43381 

12067 

72.2 

12571 

71.0 

11 

28091 

3360 

88.0 

3230 

88.5 

18 

28850 

17064 

40.8 

12511 

56.6 


From the above table it is clear that dual control sometimes yields larger 
cost reductions, and at times both the controllers perform equally good. Thus 
each run needs to be analysed individually in order to discover whether the 
prevalent conditions are congenial to the use of the dual control. The issues of 
controllability of the true plant and the controllability of the estimated plant 
are of concern to the analyst. These are discussed next. 

Run Number 1 

Referring to Figure 29, one can clearly see that the dual control improves 
over the cautious control's performance for the region 16 _< k _< 28, by probing 
effectively during 13 _< k _< 15. The rest of the plot is similar to that of the 
cautious controller. 

Let us first consider the cautious controller. From Figure 27 it is seen that 


34 




the determinant of the estimated parameter transfer matrix is zero between the 
time steps 11 and 25* This corresponds to the case of the ratio of smaller to 
the larger eigenvalues being zero in Figure 28 (also a Run 1 case). The controls in 
Figures 17 and 18, however, are still bounded because of the caution terms present 
in the control design. From figures 21-26 we see that in Region 1 02 is estimated 
with the wrong sign. This leads to a large contribution from the cosine component. 

In Region 2, 9^ has a wrong sign. In addition to this the controls are almost turned 
off and the sine component has a major share in the vibration. In Region 3, 0^ is 
estimated well but it is close to zero. Both the sine and the cosine components 
contribute here. 

For the dual we see from Figures 49, 50 that the cosine component contributes in 

Region 1 and both the sine and the cosine components contribute in Region 3. The 

controller probes the system and the control values in Figures 51, 52 are non-zero. 

Next we refer to Figures 53-58. 0^ is estimated again with the wrong sign in Region 

1. 9^ is estimated correctly between the time steps 10 and 13 and between 34 and 35 

but its true value is close to zero. In Region 3, 0^ has the wrong sign. In Region 

3, 0 £ is close to zero again. It is clear from Figures 59, 60 that the determinant 
o 

of the estimated parameter matrix is not zero between the time steps 11 and 25. 

The main improvement is in Region 2 and this leads to a reduction in the 
average cost from 20215 (Cautious) to 15762 (Dual). 

Run Number 2 

Referring to Figure 30 , one sees clearly that both the dual and cautious 
controllers perform poorly between the time steps 6 and 8 and between 15 and 25. 

In these regions the vibrations are more than that obtained by applying no control. 
For this run, reference is made to Figures 61-68 . There is a large contribution from 
the cosine component between the time steps 15 and 25 (Fig. 61). 

It is observed clearly from Figures 69-72 for both the controllers, the true 
transfer matrix is singular, corresponding to an uncontrollable mode. The parameters 
© 2 , 9^ are responsible for this. The estimation of the parameters is reasonably 

good. Under this situation applying no control is the most judicious choice. Any 
other controller could make the situation worse, which is clearly demonstrated here. 
At other time steps, when the plant is, in fact, controllable, both the controllers 
perform well. This uncontrollable situation can be handled by considering a reduced 
order model or by applying some switches on the state and control weights. This will 
be discussed later on in detail. 
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Run Number 11 


From Figure 39 it is clearly seen that both the cautious and the dual controllers 
are successful in bringing down the vibration to a satisfactory level. Both of them 
reduce the vibration by 88% of that of no control. 

Both the controllers perform well because the plant is controllable. The contribu- 
tions from the sine and the cosine components are reduced substantially. The detailed 
performances are portrayed in Figures 73-80. The determinants of the matrices composed 
of the true and the estimated parameters and the ratio of their eigenvalues are 
plotted in Figures 81-84. They are far from zero. 

Run Number 18 

A comparative plot showing the performances of the no control, cautious and the 
dual controller is given in Figure 43. Poor performances are observed between the 
time steps 9 and 20 and again between the time steps 29 and 33. In the first 
region, at time step 11, the performances are worse than that of the no control. 

Beyond time step 11 until time step 16, the dual controller improves the situation, but 
the cautious still performs poor. The detailed performances are given in Figures 85-92. 

The determinants of the matrices composed of the true and the estimated para- 
meters and the ratio of their eigenvalues are plotted in Figures 93-96. For the 
cautious controller, the estimated parameter transfer matrix has a small value for 
its determinant between the time steps 11 and 16. The dual controller starts probing 
earlier around time step 9 (.Fig. 87, 88, 91, 92) and its estimated parameter 
transfer matrix has a non zero value for its determinant during the same time steps. 
Moreover, during the time steps 16 and 20 the true plant is close to uncontroll- 
ability and nothing can be better than no control. 

From the detailed study discussed above one may conclude the following: 

1) Both the controllers behave poorly whenever the true system is uncontroll- 
able. The controllers have no information, whatsoever, about the present 
or the future controllability of the plant. In such a situation, the 
correct diagnostic is to apply no control at all. 

2) The cautious controller behaves poorly if the estimated parameters 

define an uncontrollable situation although the true system is controllable. 
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In such situations, the dual controller can offer significant improvement 
over the cautious controller, by probing to better estimate the parameters. 

Optimal Control Requirements for the Buns 1, 2, 11, 18 

In this section the cautious and the dual controllers are used on the linear 
model whose unknown parameters are time varying. The stochastic controllers are 
used in conjunction with an estimator, which supplies the controllers with the 
parameter estimator. An optimal controller, on the contrary, assumes perfect know- 
ledge of the plant parameters and attempts to control the plant. In this subsection, a 
study is made on this optimal controller. Plots of the two required optimal controls 
are given in Figures 97-104. While plotting, the controls are passed through a 
hard limiter with limits between -10°' and +10°. When the true plant is uncontroll- 
able, an optimal controller demands an infinite amount of control and thus it 
hits the boundary of +10° or -10° . From these plots , it is evident that for all but 
Run 11, the plants are uncontrollable at some time during the 40 time steps. 

Effect of switches on the control weights R 

It has been discussed earlier that in uncontrollable situations the proper 

diagnostics is to apply no control at all. The control can be switched off by 

exercising exceedingly large caution or by putting large control weights R in 

the control design. In Run 2, the region between time steps 15 and 25 is known 

to be uncontrollable. Thus between these regions the control weights are increased 

4 4 4. 

from R = diag (0,0) to R = diag (.10 ,0) and R = diag (10 , 10 ) , keeping the state 

/| t 

weights Q = diag (1,1) as before. The case *R = diag (10 ,0) switches off the 

4 4 

control u^ and the second case 'R = diag (10 ,10 )' switches off both the controls 
u^ and U£. The total and the individual costs are plotted in Figures 105-107. This 
switching is quite successful in reducing the vibrations in uncontrollable 
situations . 

Effect of switches on the state weights Q 

During uncontrollable situations, it is impossible for the two controls to 
affect both the states. Nevertheless, it may be possible to handle one state only 
at the expense of allowing the other state to run free. This can be done by 
choosing properly the state weights Q. For Run 2 again, Q is chosen as diag (l,.l) 
and diag (1, .01) for R = diag (0,0) and R = diag (,01, .01) . The total and the 
individual costs are plotted in Figures 108-113. The cases Q = .diag (l,.l) and 
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and Q = diag (1,.01) put less constraint on the second state. Thus these switches 
reduce the first state, allowing the second state to go free. The performances 
hardly differ for the cases R = diag (0,0) and R = diag (.01, .01). 

Effect of Initial Covariance of the Parameters 
on the Controller's Performance 
(first method of initialization) 

Two methods of initialization have been discussed and their performances 
have been analysed in detail earlier. This study refers to the first method of 
initialization where the Kalman filter is initialized with parameters taken from 
a random number generator with an initial covariance. Both the cautious and the 
dual controllers are used on this plant. The parameters are slowly varying with 
time (process noise of 10%). A Monte Carlo simulation of 100 runs is made for 
40 time steps. The average cost is computed over all runs for each time step. 

Three choices are made for this initial covariance. The normal initial variance 
specified on the plots is the square of the magnitude of the true initial parameters. 

The average performances of the no control, cautious and the dual are plotted in 
Figures 114-117* The dual controller offers improvement in the case of large 
initial variance (Figure 116). It has always an initial jump for this large initial 
variance (Figure 114,116). This is not observed when the initial variance is small 
(Figure 115) . This initial jump can be avoided by using a cautious controller in 
the beginning and switching to the dual after time step 2 (Figure 117). 

5*7 Simulation Results for the Nonlinear Model 

The nonlinear model describing the longitudinal hub vibration model is used 
with the global linear adaptive cautious controller of 112]. The alternate form 
of the Kalman filter which retains better positive definiteness of the covariance 
equation is used. The exponential weighting form of the equations is also used 
with the forgetting factor X set equal to .99 to discount past data. The initial 
covariance is taken as a large quantity 250000 (compared to approximately 10000 for 
the original covariance) to better account for nonlinearity. The graphical descriptions 
of the cost function of Figures 1—3 indicate the presence of multiple minima solutions. 
The plots can be broadly divided into the following regions: 


Region I : 0 _< u^ _< +2, 

-2 < u 2 < 0 

Region II : 0 _< u^ _< 2 , 

0 _< u 2 < 2 
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Region 

III : 

-2 . < u 1 < 0 , 

0 < u 2 

< 2 

Region 

IV: 

-2 < u 1 < 0 , 

-2 < u 2 

jc 0 


These definitions of the regions provide easier interpretation of the per- 
formance of the controllers. Figures 118-122 show the convergence plots of the 

cautious adaptive controller for 100 time steps. Figures 118-120 are for the 
-5 -8 -4 -4 

case Q = diag (10 , 5x10 ) and R = diag (10 ,10 ) . These weights primarily 

concentrate on reducing the state x^ while allowing less restriction on the state 
x 2 . There are more than one occasion before time step 80 when the cost increases 
sharply. The controls are allowed to move within the limits of +2. At these 
points the control is operating in Region II. In this region state x^ is highly 
nonlinear and appears to possess a saddle point. At a saddle point the second 
gradient of a function is zero. This corresponds to an uncontrollable situation 
as is clear from the equations of [17]. Figures 121-123 discuss the case for 
which Q = diag (1,1). and R = diag (0,0) . Here both the states x^ and x^are 
equally weighted. In this case convergence occurs rapidly with the controls 
going to Region I. From figures 1,2 it is clear that the two individual 
states x^ and x^ are well behaved In this region and a fast convergence occurs. 

5.8 Simulation of the Scalar Model using the Cautious, Dual 

and the Dual Adaptive Control based upon Sensitivity Functions 
(Appendix A and B) 

i 

A new adaptive dual control solution based upon the sensitivity functions of 

the expected future cost is derived and analysed in the Appendices A and B. The 

controller design and its performance on a scalar model are given in Appendix A. 

The performance is compared with that of the cautious controller and the first order 
dual controller. This section discusses in further detail the results shown in 
Appendices A and B and includes several other figures from the 100 Monte Carlo 
runs for both the constant parameter and the time varying parameter cases. 

Time Varying Parameter Case (Example a) 

The details of the simulation model used are presented in Appendix A for 
the time varying parameter case. Run numbers 7 and 14 are discussed in the Appendix 

A. Runs 2 and 21 are discussed here. The cautious control is shown by the circle 

symbol, the first order dual by the triangle, and the second order dual by the plus 
symbols. Figures 124-126 describe this run. The second order dual performs better 
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than the other two right from the start. Figures 127, 128 gives the time history of 
the control for runs 7 and 14 respectively. Figures 129-131 describe Run 21. The 
nature of the performance of all the controllers is the same but in most occasions 
the second order dual (denoted by the plus symbol) performs better than the other 
two (see Tables 2, 3, and 6). 

Constant Parameter Case (Examble b) 

In this case (Section 4) the true parameter is close to zero (i.e., b(0)=.05) 
but constant. This corresponds to the uncontrollable case. A control magnitude of 
-20 is required to handle this situation. Runs 26 and 80 are discussed in the 
Appendix A. Figures 134 and 137 show the time history of the control for Runs 26 

and 80. Figures 132 and 133 describe Run 18. Here the first order dual tends to 

go unstable in the 11 to 15 time step and the cautious is slow in its convergence. 

Figures 135 and 136 give the cost and the control time history of Run 44. Both the 

duals work better than the cautious. In all runs of this constant parameter case 
the new dual shows better results than the other two. It always goes to the right 
direction of estimation by properly probing from the start (see Tables 4, 5, and 7). 
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6. CONCLUSIONS 


Helicopter vibration can be effectively reduced by applying adaptive 
control techniques. Although adaptive controllers often show excellent vibration 
reduction, it has been shown that there exist certain conditions which yield unsat- 
isfactory behavior and thus require improvements. It is shown that the existing 
cautious controller often exhibits problems like slow convergence, turn off 
phenomenon, and instability. Also, nonlinearity in the helicopter model is shown 
to have a significant effect on the HHC convergence behavior. The performance 
of the cautious controller has been evaluated based upon 100 Monte-Carlo simulations 
of a linear time varying multivariable model representative of helicopter vibration 
during maneuvering flight conditions. The cautious controller has been shown to 
yield unacceptable reduction in vibration whenever the determinant of the estimated 
parameter model is near zero. This results in an tin controllable model. The first 
order dual controller of [13] avoids this situation by probing and estimating the 
parameters better. It however is still deficient under certain conditions and 
sometimes yields poor results. 

The problems of the previous controllers are overcome by the new adaptive dual 
control solution developed in Appendix A. -Tfie second order dual controller takes 
into account the dual effect better by performing a second order Taylor series 
expansion of the expected future cost. It is shown to modify the cautious control 
solution by introducing numerator and denominator correction terms. These three 
controllers have been evaluated on both a simple scalar constant parameter model 
and a time-varying parameter model representative of maneuvering flight. In 
each, case 100 Monte-Carlo simulations are used. The new dual controller improves 
upon both the constant and the time varying case and provides improved convergence. 
In the case of the constant parameter model, whenever the unknown parameter is 
close to zero (i.e. near singular}., the cautious and the first order dual control- 
lers have demonstrated slow convergence and turn off. The new second order dual 
controller consistently demonstrates faster convergence. It avoids problems of 
turn off, slow convergence and always goes toward the right direction of estimation 
by properly probing. It has the potential to be used on a multivariable 
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model and the detailed s ens it iyit i es have. been presented 

in this report. It is yet to be implemented on this multivariable model. Moreover, 
its properties and the behavior of the correction terms that enable better performance 
in the scalar case are not yet fully understood. Computationally, it is complex 
and may be difficult, to use in real time. However, investigation Of its properties 
is thus warranted in order to develop a practical implementation. 

In addition to the above linear simulation studies, the global linear adaptive 
cautious controller has been used on a nonlinear model describing helicopter vibration. 
Nonlinearity has a significant effect on the convergence behavior. Multiple minima 
solutions can exist and the algorithms are slower in convergence and can be unstable. 

To accommodate the nonlinearity, the initial covariance on the parameter estimates 
is taken as a large quantity. This accounts for the nonlinearity by introducing 
more caution. Reduction of vibration is possible in most regions of the nonlinear 
surface. Vibrations cannot be reduced when the nonlinear surface possesses a 
saddle point. At a saddle point the second gradient of the cost function is zero. 

This corresponds to an uncontrollable situation and can result in algorithm 
divergence. 

The linear and nonlinear simulation studies investigated in this report clearly 
demonstrate the need for further research to better understand the convergence 
properties of the adaptive controller for reduction of helicopter vibration. In the 
present studies the analytical ground work has been presented. Further analytical 
work and simulation is required to fully understand the properties of the adaptive 
controller for helicopter vibration reduction. 
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APPENDIX A 


DUAL ADAPTIVE CONTROL BASED UPON SENSITIVITY FUNCTIONS 


Anew adaptive dual control solution is presented for the control of a class 
of multi-variable input -output systems. Both rapidly varying random parameters 
and constant but Unknown parameters are included. The new controller is based 
upon a on-line Newton type algorithm which is shown to result in a controller which 
modifies the cautious control design with a numerator and denominator correction. 

The new controller is shown to depend upon sensitivity functions of the expected 
future cost. A scalar example is presented to provide insight into the properties 
of the new dual controller. Monte-Carlo simulations are performed which show 
improvement over the cautious controller and the Linear Feedback Dual Controller 
of 10 (13) and (14). . 

A1 . INTRODUCTION 

Multi-variable systems which are characterized by uncertain parameters with 
large random variations are a difficult challenge for most control design techniques. 
The assumed random nature of the parameter variations often precludes the use of gain 
scheduling (non adaptive) control design. Stochastic adaptive control theory 
provides a principal design approach for systems of this type. Exact solution of 
the stochastic problem with unknown parameters requires solution of the Stochastic 
Dynamic Programming equation and this is not feasible for practical implementation. 
The solution is known to have a dual effect [13, 14] that can be used to enhance 
the real-time identification of system parameters as well as provide good control. 
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Many suboptimal dual solutions have been suggested [11,13,14,27-32] . The various 
approaches which have incorporated this dual property can be loosely divided into 
two classes. In the first class [28-31] , the optimal control problem is 
reformulated to consist of a one-step ahead criterion to be minimized, augmented 
by a second term which penalizes the cost for poor identification. This approach 
is attractive due to the analytical tractability of the solution; however, the 
solution is based on a one-step criterion and does not fully exploit the dual 
property of a multi-step solution. Padilla and Cruz [34] give a dual control solution foi 
such a plant by minimizing the control objective function subject to an upper 
bound in the total estimation cost. Their objective function includes a standard 
control objective function and also a second constraint term which, reflects the sen- 
sitivity of the parameters to the state of the system. Thus the solution adjusts itself 
to exercise better estimation for such sensitive parameters within the upper bound. 

The second class [21,.32,33] utilizes the stochastic dynamic programming equation 
directly and performs linearization of the future cost in order to obtain a solution. 
Previous control solutions among this second class require a numerical search pro- 
cedure which poses difficulties for a practical solution for on-line control for 
multivariable systems. 

A linear feedback dual controller was presented in [13,14] based 
upon a linearization of a two-step criterion and found to offer some improvement 
over the non-dual cautious control based upon a one-step criterion. The results 
were based upon a simulation model with constant but unknown parameters. Although 
the dual control offers some improvement over the cautious controller the improve- 
ment is not significant for most practical applications where the system contains 
constant parameters and the objective is to control in steady state operation. 

However, for random parameter variations, dual control can sometimes offer signifi- 
cant improvement over non-dual controllers [28,32] . The approximate dual control 
in [13,14] is attractive due to its simplicity (it- is comparable to the 
cautious control design in algorithm complexity and does not require numerical 
search). The objective of the present study is to evaluate the cautious controller 
and the. approximate dual controller of [13,14] for- large random parameter 
variations modeled as a random walk. Monte-Carlo simulations are performed and 
conditions quantified under which the dual controller offers significant improve- 
ment over a non-dual cautious controller. 
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The approximate dual control of [13,14] although offering a reduction 
in the average cost is found to be unacceptable in many cases. This is attributed 
to the sensitivity of the expected future cost whenever the system is characterized 
by limited controllability. An extension of the linearization procedure of 
[13,14] is presented to account for this sensitivity. The new dual controller 
inherently includes a robustness property in that the controller accounts for 
sensitivity of the expected future cost due to parameter estimates and their uncer- 
tainty. Simulations are presented which show the improvement of the new dual 
controller over the cautious controller and the approximate dual controller 
of [13,14] . The new dual controller uses a Newton type search procedure 

and is developed for multi-variable systems. One of the main advantages of the 
new dual control presented herein is that it modifies the cautious controller with 
a numerator "probing" term and denominator correction term. Although the new 
dual control is still considered too complex for practical implementation,, 
the structure of the control solution is in a form which permits practical design 
changes to the cautious controller to include the dual properties. 
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A2 . 


PROBLEM FORMULATION 


The multivariable system under investigation is 

x(k+l) = c(k) + B(k) u(k) (A2.1) 

where c(k) is an unknown vector and B(k) is a matrix of unknown parameters. The 
unknown elements of c(k) and B(k) are denoted as 0(k) with covariance matrix P(k). 
These are represented by a discrete random model 

0(k+l) = A0(k) + v(k) (A2.2) 

E(v(k) ) =0 and E(v(k)v'(j)) = V 5, . (A2;3) 

kj 

The measurement equation is 

y(k) = x(k) + w(k) (A2.4) 

where 


E(w(k) ) = 0 and E(w(k)w’(j)) = W 6 kj (A2.5) 

E(w(k)v' Cj) ) = 0 

and x(k) , y(k) being n dimensional vectors. The control criterion to be minimized 
is the expected value of the cost from step 0 to N 

J(0) = E{C(0)>e|e x’(k) Q x (k) + u'(k-l) R u(k-l)| (A2.6) 

where N = 2 for the two step ahead criterion. 


A3. APPROXIMATE DUAL CONTROLLER FOR TWO STEP CRITERION 

The minimization of (A2.6) with respect to u(0) and u(l) subject to (A2.1) - 
(A2.5) is obtained from the Stochastic Dynamic Programming equation [19,20] 

J*(k) = min E{C(k) + J*(k+1) |Y k } k = N-l, . . . , 1, 0 (A3.1) 

u(k) 

where J*(k) is the "cost-to-go" from k to N and Y is the cumulated information at 
time k when the control u(k) is to be determined. For N = 1, eq. (A3..1) is 


J*(0) = min E {x’(l)Qx(l) +u'(0) Ru(0) + J*(l) |y°} (A3. 2) 

u(0) 

where J*(l) is the optimal cost at the last step and is obtained by minimization 
of J (N-l) for N = 2. Assuming diagonal Q - diagCq^) this results in [,13,14] 

J*(l) = c'(l)QcCl) + l q„P £ Cl) 

1=1 * C 


- [c' (l)QB(l) + Z 

1=1 


n 


q £ P cB (1)] C B ' (l)QB(l) + 




+ R] 


- 1 . 


[B’Cl)Qc(i) 


+ j 1 i 


(A3. 3) 
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and 


u*U) - -[B’(l)QBCl) + Z q £ Pg(l) + R] _1 [B' (l)Qc(l) + \ <i.£ P £ c CD ] (A3 - 4) 


where 


l 

p (i) = 


pf(i) 


p i u) 

P B (1) 


(A3. 5) 


PCI) is the expected value of (9(l))^for time step 2 given measurement y Cl) 
at time step 1. The index Z. is used to represent the row number in Eq. (A2.1) and 

t 

P Cl) is the associated parameter covariance. 

. % 

A 

The parameter estimates 0(1) and covariances P (1 ) are obtained from the Kalman 
filter. Since W is diagonal one can decouple the estimation. Then 


whe re 


0^(1) 

= ?(0) + K £ (l) v^(l) 

C (A3. 6) 

¥ 1} 

= p¥o)H’( 1) [H(l)P' e (0)H , (l) + W^r 1 

- (A3 . 7) 

? l a) 

« p £ (o) - k^i)h(i)p¥o) 

(A3 . 8) 

p^d) 

= AP^(1)A’ + V 

(A3. 9) 

V 1 ) 

= y e (l) - H(1)0' £ (O) 

... (A3. 10) 

H(l) 

= [1 u T (0) ] 

(A3. 11) 


0*U) « [c £ CD b £ (1)] t 


l = 1,2, . . .n row of B 


(A3. 12) 


As discussed in : [13,14] J*(l) is a nonlinear function of the 

A 

parameter estimates 8(1) and covariances P(l) and thus a linearization was per- 
formed. In [14] a scalar formulation was presented and a first order lineari- 

^ 2 

zation was performed about the nominal parameter estimate squared (_0(_Q)) and nominal co 
variance P(l) , Also in [13,14] the vector case was presented and linear- 

ization. to first order performed. To more accurately account for the dual effect 

A 

a second order Taylor Series expansion is presented about 0(0) and a first order 
expansion about the nominal covariance P(l) . In addition (as will be presented sub- 
sequently) the covariance PCI) will include a linearization to second order in 
u(0) . In [13,14], PCI) was linearized to first order. It is believed 

that linearizations to second order are necessary to better account for the non- 

A 

linearity in PCD and 0(1) of Eq. (A3. 3) and in u(0) of Eq. (A3. 7) and: (A3. 8). In 
addition a nonlinear Newton algorithm is used in the second order approximation. 
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Linearization of Eq. (A3. 3) about -the nominal 6(1) = 6(0) and P(l) using the 
nominal u(,0) results in 

/v ^ 2 

j*U) = J*[i, 0(o), pci)] + - x J * (1) [ 9 (D _ 6 ( 0 )] + i[0(i)-0(o)] f ^j^- [e(i)-e(o) ] 

ae(D 30 (i) 


n n n 

+ Z l Z 


3J*(1) 




7 — t p ; 4 a) -?± i ci)] (A3 - 13) 
M i-1 j-1 3P. ,(1) * 3 ’ J 

1 jJ 


,th 


where the superscript Z represents the covariance matrix associated with the Z 

row of parameters and P . .(1) is the i-j th element of the covariance matrix PCD 

J 

Eq. (A3. 6) is rewritten as 


r 


(1) - ?(0) = 'Kp(l) V £ (l) , Z = 1, 2, ... n 


(A3. 14) 


Using (A3. 14) the expected value of (A3. 13) is 


E [ J*(l) jY°] = J*[l, 6(0), P(l)] + \ tr{ '^4*— K (D E{v(l)v' (1) }y°} K* (1) ] 

ae z (i) 


n n n 
ZEE 


[p £ . (l) - pf . CD 1 

1=1 i=l j-l 3P . .(D 1,J ,J 

J x ,3 


(A3. 15) 


Using the innovation covariance 

1 »T/1 \ T-» Zf 


e(v £ (1) v£C1) |y°} - H(1) P (0)H'(1) + W £ 

and (A3. 7) and (A3. 8), Eq. (A3. 15) can be written as 


(A3. 16) 


n 


E[J*(1) |Y ] = J*[l, 6(0), P(D] + E E Z 

Z= 1 i=l j=l 


1 

2 


9j*(l) 


86 i (l) 38^1) 




. 3J*(1) 

„ Z 

The expected future cost 


[p £ .(D - pf .(D ] 

h3 *>3 


(A3. 17) 


and 


is shown to be a function of the pre- 

£ 3J*(1) 

dieted covariance P. , £L)wit-h a multiplier given by the sensitivity » 

3p. .CD 

^ 9J*(D Z 

— — - — - . Since the covariance P. . CD depends on the control u(0) the 

'• 1,3 - 

96 .(1) 36.(1) 

control has the dual effect. It should be noted that the importance of the dual 
effect depends upon the sensitivity of the expected future cost with respect to both 
the covariance and parameter estimate. 
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The optimal control u(0) can be computed by minimization of (A3. 2) using 

t 

(A3. 17). Since P. (l)is nonlinear in u(0) a numerical search procedure is 

i >j 

required. This is accomplished using a second order linearization in u(0) . 

Thus Eq. (A3. 8) is linearized to second order about the control u^(0) , which 
is in the vicinity of the optimal control. 


p c a) 


p £ .CD + 

x ,3 


a^ci) 

3u(0) 


2 a 

T T T 3 P t - (1 > 

T [u(o)-u(o) ] + ftucoy-^coir — ^ — 
u(o) 2 a u 2 (o) 


[u(O)-uCO) J 

u r (0) 

(A3. 18) 

The expected future cost as given -by (A3. 17) and (A3. 18) is quadratic in uCO) 
and thus a closed form solution u*(0) is obtained by minimization of (A3. 2) . 

The optimal dual control u*(0) can now be computed from (A3. 2) using (A3. 17) 
and (A3. 18). It is obtained by solving 


— E{x'(l)Q x(l) + u * CO) Ru CO) + J*(.l) |Y°} = 0 (A3. 19) 

3u(0) 

The optimal u*(0) is thus 


u"(0) = 


- [B ' (0) QB(0) + Z (q^O) + V + R]~ 1 [B* (0)Qc(0) + Z + * £> 1 

1 1 


where the matrix and the vector f^ are 


n 1 

/ 3J*(1) 

1 3 

3J*(1) \ 

a apfu) 

A . h3 

A 2 1 

3=1 


2 ''L 

2 39pl) 

30^(1) / 

3u(0) 3u(0) 


(A3. 20) 


(A3. 21) 

^(0) ,8(0) ,P(.l) 


* ® 1 / 3J*(1) 

f (> = L Z — j J 

^ i-1 j==l 2 \ SP -6 (1) 


1 3 

2 


3J*(1) 




~ 2 * L 

3 0.(1) 30.(1) 
i J 


3pf (1) a 3pf .(1) T 
A U u 1 




3u(0) 3u(0) 3u(0) 


(0) 


(A3. 22) 

u X (0) ,6(0) ,P(1 


Initially the nominal value of u(0) is computed from (A3. 20) with and 
f ^ equal to zero. Then a gradient search is performed until in the vicinity of 
the. optimal u (0). Then (A3. 20) - (A3. 22) are used until convergnece 

is achieved. This iteration procedure is essentially Newton’s method for minimization 

of a nonlinear function. The gradient search is used because the stochastic cost 
in (A3. 2) being minimized is a high order nonlinear equation and the gradient * 

procedure is used until u^(0) is in the vicinity of the minimum before switching 
to the Newton method. The nominal covariance P (1) is computed from (A3. 7) - 
(A3. 11) with u(0) = u(0) , The sensitivity (partials) in (A3. 21) and 
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(A3. 22) of the cost J*(l) Are computed from partial derivatives of J*(l) (Eq. (A3. 3)) 
and F^(l) (Eq. (A3. 7) through (A3. 9)) evaluated at the nominal. The partials of the 
covariance are evaluated at u^(G) which is evaluated at the previous iteration I. 

The approximate two-step ahead dual control of Eqs. (A3. 20), (A3. 21); and (A3. 22) 
can be interpreted as a modification to the cautious controller by the terms F^ 
and fj>. These terms depend upon the sensitivity of the future nominal cost J*(l) 
with respect to the parameters 0^(1) 07(1) for all i, j and their covariance (1) for 
each row Z of parameters. Whenever these sensitivities are large the terms and 
f^ will be significant (that is the dual effect will be important). Thus the 
sensitivities take into account in the control solution the sensitivity of the 
nominal future cost due to parameter variation and uncertainty. The larger this 
sensitivity the more important will be the dual effect. 

The resulting dual controller (A3. 20) exhibits a robustness property with 
respect to parameter variations and uncertainty of the future cost by including 

a term which appears in the denominator of the dual controller. In 
addition, a probing term also appears in the numerator. 

A4. SCALAR EXAMPLE WITH ONE UNKNOWN PARAMETER 

To further understand the dual control solution a scalar example with one 
unknown parameter b is presented (a multi-variable simulation is currently under 
development). The approximate dual control solution for the scalar case from 
Eqs. (A3. 20), (A3. 21) and (A3. 22) using Q = 1, R = 0, is 


u 5 (0) - - 


b(0) c + f £, 




(.0) +? b (0) +F ^ 

I 3J*(1) 


1 3 2 J*(1). 


3P b CD 


3b Cl) 3b(l) y 


3 2 P b U) 
3u(0) 3u(0) 


(A4.1) 


(A4.2) 


■P 


1 

2 


3*J(1) 

3P b (D 


1_ 

2 


2 * 

rj (i) 

3b(l) 3bCl)y 


V1)_ 

3u(0) 


8 2p b (1) 


3u(0) 3u(0) 


^(0) 


(A4.3) 


3J*(D 

3P b (l) 


cV(0) 


b(0) ,P b (l) 


(b 2 (0)+P b (l)) 2 


(A4. 4) 
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(A4.5) 


aV(D 


3b(l) 3b(l) 


b(0),P b (l) 


- 2c 2 P b (l) 


P b (l)-3b' (0) 

. 

(b 2 (0)+P b (l)) 3 


3P b (1) 

3u(0) 


u X (0) 


2? b (0)W u* (0)a 2 
(P b (0)u l2 (0)+W) 2 


(A4.6) 
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3u(0) 3u(0)j 


2 

r I 2 

W-3P, C0)u CO) 

. - -2P (0)W 

uho> b 

b 

i 2 3 

L(p (o)u 1 co)+w) - 


(A4.7) 


where the nominal u(0) and P, (1) are 

b 


u(0) = - 


b(0) c 


b (0)+P b (0) 


(A4.8) 


P, Cl) - a- <Y(0) _ 


p 2 (0)u 2 (0) | 


V 7 _o / 

P b (0)u / C0)+W ) 


+ V = 


a P, (0)W 
0 

P b (0)u 2 (0)+W 


+ V 


(A4.9) 


The parameter estimate b(0) and P b (0) are computed using data up to 
k = 0 (i.e. y(0) ) . 

The future cost evaluated at the nominal is 


J*(l, b(0) , P(D) = c 2 - c 2 


^2 

b(0) 


b C0)+P b (l) 


(A4.10) 


and the expected future cost based upon the linearization of Eq.' CA3.17) 

s\ 2 * ^ 

e{j"( 1 ) |y° } = j*Ci, bC 0 ), PCD) - \ CP. CD -p. CO) -v) + Cp,CD-p, 

2 3b 2 Cl) b 3 p, (l) b 1 

b 


CA4.11) 


CD) 
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A4.1 Evaluation of the Cautious Controller 

The performance of the cautious controller can be evaluated using (A3. 2) 
with u(0) evaluated at the nominal 


J(0) = [E{x 2 (1) |y°} + E{J*(1) |Y°>] uC0)= - C0) ----- (A4...12) 

The first term in Eq. (A4.12) represents the expected cost at k = 1 and the second 
term in Eq. (A. 12) represents the expected future cost at k = 2 using the cautious 
control at k - 2 (i.e. u(l)) and using the cautious control at k = 1 (i.e. u(0) = 
u(0)). Eq. (A4.12) is evaluated using data Y°. 

Eq. (A4.12) is evaluated for the scalar example using (A4.10), (A4.ll) and 
(A4.7) - (A4.9), which results in 


JCO) = c 2 


l?(0)c 2 , 2 

1 . c 

b^(0)+P b (0) 


2 b 2 (p) 1 9 2 J*(1) 

^ ' r\ Aa 

b Z (0)+P b (l) Z 9b Z (l) 


P 2 (0)u 2 (0) 

P u (0)u 2 (0)+W 

D 


(A4.13) 


3 J ( 1) 

where ^ '■ — is computed from (A4.5) . 

3d ■(!) 

The last term in (A4.ll) ia zero since P, (1) evaluated at the nominal 

D 

control (i.e. cautious control) equals P^(l) . 

The first two terms in (A4.13) represent the average cost at step k = 1 

and the last three terms represent the expected future cost at k = 2 using the 
cautious control. 

Eq. (A4.13) can be used with a simple example to demonstrate when the cautious 
control is expected to behave poorly. 

Assume a scalar example with one unknown b parameter and let 

A 

b(0)'= .05 , P(0) = .5 , a = 1.0 (A4.14) 

V = .1 , W = .1 , c = 1 

The expected cost at k = 1 and k = 2 is computed from the nominal, u(0) , 

2 * 

P, (1) and — 1 — — which yields 

D 9b Z (l) 


and 


u(0) = -.1 , P b (l) = .575 

J(0) - c 2 + c 2 , c = 1 


7 iV 

(1) - = -3.47 
9b Z (l) 


(A4.15) 

(A4.16)- 
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Thus the cautious control applied at k = 0 results in no reduction in the 
cost at k = 1 due to large uncertainty P(l) and also no reduction in the future 
expected cost since u(0) is small and no improvement in parameter accuracy occurs 
at step k = 1. 

A4.2 Evaluation of the Dual Controller 

The dual controller given by (A4.1) - (A4.9) can be evaluated by computing 
the average cost of (A4.12) using (A4.7) - (A4.ll) and the covariance 

P h (0)W 

P Cl) = + V (A4.17) 

P t (0)u (0)+W 
b 

The expected future cost (A4 .11) reduces to 


E{j*(l) !y°} 


« c 
u*(0) 


2 

c 


b 2 (0) , 

b 2 (0)+P b (l) 


1 3 2 J*(1) 

2 ipot 


2 *2 
P b (0)u Z (0) 

P, (0)u* 2 (0)+W 
b 


9J*(1) 

»b a) 


2 *2 
P b (0)u - (0) 

P, (0)u* 2 (0)+W 
b 


P 2 (0)u 2 C0) \ 
P b (0)u 2 (0)+W J 


(A4.18) 


and the total expected cost at k = 1 and k = 2 using (A4.12) is 


j 5 t (o) 


e{x 2 (d |y°: 


e{j*(i) 


;y°}| 


* 

u (0) 


|u*(0) 


wnere 


(A4.19) 


E {x (.1) 


Y°} 


= c + 2b(0)u (O') c + 


^2 

(b (0) 


P b (0))u* 2 (0) 


(A4.20) 


u (0) 

Examiuation of (A4.18) shows that the dual control can reduce the expected 

future cost over the cautious control since the last two expressions in (A4.18) 

*2 2 

can be negative if u (0) > u (0) . Thus the dual property can have a desirable 
effect on the future cost. 

The cost J (0) is computed using the scalar example previously discussed 
for the cautious controller. A search procedure is used on (A4.19) using 
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(A4.18) and (A4.20) with the parameter values from (A4.14), and u (0) 

is iterated until in the vicinity of the minimum yielding 


ai*(i) 

3P b (D 


u(0) 


0075 

1 


2 * 

Q (1) 
3b Cl> 


= -3.47 
u(0)=-.l 


9P b Cl) 

3u(0) 


u X (0)=- 


382 

6 


3u 2 (0) 


U X C0) 


+1 . 0 
=-.6 


= . 87 , = . 85 (A4..21): 


Eq. (A4.21) was evaluated in the vicinity of the optimal u (0) - -.6 and ?..(!) = .278. 

b 

The optimal control u*(0) is from Eq. (A4.1) using unQ) = -.6 


u*(0) 


b(0) c + .85 
b 2 (0) + P b (0) + .87 


c - 1 


(A4.22) 


= -.62 

Eq. (A4. 22) shows that u*(0) is considerably different than the cautious control 

u(G) - -.land is a result of large values of F» and f n which in turn are due 

to iarao values of ^ 

3u (0) 3t> (l) 


The corresponding future expected cost using Eq. (A4. 
E{j'"(l) |y°} 


is 


u”(0) 


„ 2 1 3 2 J (1) 

" C 2 ~ k 2 

z 3b (1) 


P 2 (0)u* 2 (0) 

P, (0)u* 2 (0)+W 
b 


= C 2 - | (3.47) (.321) C 2 


~ 2 c c 7 2 

- c - *557 c 


- -442 c 2 , c - 1 (A4.23) 

The result of this example shows that the dual control of Eq. (A4.22) reduces 

2 

the expected future cost to 44% of the original c with no control. The cautious 
control resulted in no reduction of the future cost. The terms responsible for the 
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and 


improvement with dual control 

3 2 J*(1) 

3b" (1) 


are the second order sensitivities 


a 2 pg) 

3u 2 C0) 


The dual control of Eq. (A4.22) differs from the cautious control by the terms 

- -87 in the denominator and f^ = ,85 in the numerator. The denominator term 

in effect provides more "caution" whereas the numerator term is an additive probing 

effect. The term F^ provides a "robustness" property in that the sensitivity of 

the future cost to parameter uncertainties as they appear in the controller 
~2 

(i.e. b (0)) are minimized. Thus anew interpretation of the dual control is 
that it contains robustness and learning (via probing) . These concepts are applic- 
able to the multivariable dual controller in Eq. (A3. 20) through Eq. (A3 .22) . 

AS. SIMULATION RESULTS 


Performance was evaluated from 100 Monte Carlo runs for the following 

controllers where b(0) was set equal to b(0) with covariance P (0) : 

b 

1. Cautious Controller 


2. A two step dual based on the first order Taylor Series expansion [13,14] 

3. The new dual controller based on the second order Taylor Series expansion. 
The above algorithms were tested for two cases: 

a) Time varying case, b(0) = ,05, P b (0). = 1.0, V = .1, c = 1.0 


W = .01 and W = .1 , a = 0. 9 

b) Constant case, with bCO) = .05, P^CO) = 1.0, V = 0, c = 1.0 


W = .01 and W = .1 , a = 1.0 

Example a 

Table 6 summarizes the results of the simulation runs. All three algorithms 
were tested on this example for two different levels of measurement noise covariance, 
W — .01 and W = .1, 100 Monte Carlo runs were performed, each of 40 time steps. 

For each run, an average cost was computed over 40 time steps and then the averages 
over 100 runs are tabulated in Tabled and Table 7 . The tables clearly indicate 
that the dual controller based on the second order sensitivity functions shows the 
lower cost. The dual effect shows a larger improvement for larger measurement 
noise (i.e. W - .1). Run numbers 7 and 14 of the 100 Monte Carlo runs were selected 


for plotting. The cautious control is shown by the circle symbol, the first order 
dual by the triangle, and the second order dual by the plus symbols. The cost and 
parameter value are plotted in Figures 138-141, It is evident that the 
second order dual improves upon the other two on the average. 
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Example b 

In this case the true parameter was close to zero (i.e., b(0) = .05) 
but constant. Table 6 summarizes the result. The average cost obtained by 
the second order dual is much lower than the other two. The second order dual 
was always found to exhibit excellent convergence whereas the other controllers 
performed poorly. In addition the new controller consistently avoided turn off 
and burst [28],.. This was an important common feature in all the Monte Carlo runs. 
Runs 26 and 80 are plotted in Figures 142 and 143 respectively, as typical examples. 

The simulation study has shown that the new dual controller improves upon 
the cost on the average. The magnitude of the improvement on the average appears 
to be relatively small for the noise levels used. However, the real advantage 
of the new dual controller is the improvement in those instances where* the cautious 
controller and the dual controller of [13,14] yields unacceptable results. 

Although the dual controller of [13,14] shows improvement over the cautious 
controller, it has been found to be unacceptable at many time points. 

A6. CONCLUSION 

A new adaptive dual control solution based upon the sensitivity functions 
of the expected future cost has been presented. This controller takes into 
account the dual effect better by performing the second order Taylor series 
expansion of the expected future cost. The form of this controller is a modification 
of the one step cautious controller. The approximate dual control of [13,14] 

not have the denominator correction term like the present one. This 

stability to the new control design. Simulation results of a scalar model 
shown the improvement obtained using the new dual algorithm. 


did 

adds 

have 
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APPENDIX B 


DERIVATION OF THE SENSITIVITY FUNCTIONS FOR 
THE SECOND ORDER DUAL CONTROLLER FOR A 
TWO STATE VECTOR MODEL 


The concept of a dual controller is introduced in Section 2.2 and a dual 
controller based upon a second-order Taylor's series expansion is derived in 
Appendix A. This controller is given by 


u*(0) = -[B'(0)Q BCO) + E (q P £ (0) + Fj + R] _1 [B’ CO)Qc(O) + 

£=1 

^ Cq^CO) + f p ] ‘ (B.l) 


where the matrix F^ and the vector are 


n 

= E 


n 1 

Z 2 
i=l J«1 


3J*(1) 

3p5 .(1) 

1 9 J 


1 3 3J*(1) 

o ''0 

Z 30*(1) 30^(1) 


3p £ ,(D 


3u(0) 3u(0) 


(B . 2) 

u X (0) ,0(0) ,P(1) 



i=l j=i 2 \ SPj^Cl) 


1 3 

2 


3J*(1) 

x 


30~(1) 30j (1) j 


3P £ .CD 

X »J 

3u(0) 


3 

3u(0) 


3P £ .(1) 

8u(0) 


u I (0) 


(B.3) 

^(0) ,0(0) ,P(1) 


This approximate two-step ahead dual control is a modification of the cautious 
controller by the terms F 0 and f 0 . These terms depend upon the sensitivity of 
the future expected cost J (1) with respect to the parameters 0. (1) 0 . Cl) , for 

a 1 -* 

all i,j and their covariance P^ (1) for each row Z of parameters. The sensitivity 

evaluations of the future covariance P. .(1) with respect to the current control 

■ 1 1 

u(0) are also necessary. These functions are derived in detail in the following 
sections of this appendix. 
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Variable Definitions 


For the sake of completeness, we may rewrite Eq. (3.3) of Appendix A here, 
assuming Q = diag (q^) , as 

J*(l) = c'(l) Qc(l) + 2 q. P*(l) 

Sb= 1 * C 


- [c’CD Q B(l) + 2 q Q P J (I) ] [S’ CD QB (1) + 

2 q £ Pg(D + R]" 1 • [B* (l)Qc(l) + 2 q^Cl) ] 

£**1 il=l 

The plant equations are described by 

x 1 (k+l) = 0 1 (k) + ©2 (k) u 1 (k) + 0. 3 (k) u 2 (k) 

x 2 (k+l) = 9^(k) + 0 5 (k) u^(k) + 0g(k) u^Ck) 

whose measurements are according to 
y 1 (k) = Xj^Ck) + w^(k) 

y 2 (k) = x 2 (k) + w 2 (k) 

with 

E(w(k) ) - 0 

E{w(k)w’(j)} = WS^ ; w(k) = (w^Ck) w 2 (k))' 
and the parameter of Eqn. (B.5) and (B.6) varying as 
0(k+l) = 9(k) + v(k) 

where 

E(v(k) ) = 0 ; E(v(k) v* (j) } = 

E(w(k)v'(j)} =0 ; v(k) = (v^k) v 2 (k) ) ' 

Following the notations of [13] we have. 


e^U) 

, B(l) = 

" ®2 C1) 
A 

63(1) 

A 

e 4 d) 


e 5 (i) 

e 6 (u 


(B.4) 

(B.5) 

(B.6) 

(B . 7) 
(B . 8) 

(B.9) 

(B .10) 
(B .11) 

(B . 12) 
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and the future expected cost. 


J*(l) = ^C^U) + P lfl (X)) + q 2 ce^(l) + P 4 4 (1)) 


— — s- (F 2 D - 2FGE + G 2 C) 


CD-E 

where P (1) is the m-n element of the covariance matrix P(l) 
m,n 

C = q 1 (6 2 Cl) + P 2 ,2 (1)) + c 1 2 C ®5 (1) + p 5 5 (1)) + r l 
D = qi (^(l) + P 3i3 (l)) + q 2 (e|d) + P 6j6 d)) + r 2 

e = q 1 (0 2 Ci) e 3 (i) + p 2 (i)) + q 2 (0 5 Ci) e 6 (i) + p 5>6 (D) 

F = q 1 ce 1 (D e 2 (i) + P 1>2 (1)) + q 2 (e 4 (i) e 5 (i) + p 4>5 (D) 
g = q^d) e 3 (i) + P 1}3 (1)) + q 2 (0 4 (i) e 6 Ci) + p 4)6 (D) 


First Order Sensitivity 


9J*(1) 

3P i,J (1) 


3J*(1) 

3p i,i a) 


= 9i 


3J*(1) 

3P l,2 Cl) 


2q 1 CFD-GE) 

(CD-E 2 ) 


9P 1,3 C1) 


2 qi CGC-FE) 

(CD-E 2 ) 


•k 

3J CD 

3P 2>2 (1) 


q^{G 2 (CD-E 2 ) - (F 2 D-2FGE+G 2 C)D) 
(CD-E 2 ) 2 


a.J*(x) 

*2,3<« 


q^{C-2FG) (CD-E 2 )-(F 2 D-2FGE+G 2 C) (-2E) } 
(CD-E 2 ) 2 


3J*(1) 

3P 3,3 (1) 


q 1 (F 2 (CD-E 2 ) -(F 2 D-2FGE+G 2 C) C} 
— — 

(CD-eV 


(B. 13) 

and 


(B. 14) 


(B.15) 
(B. 16) 

(B.17) 

(B.18) 

(B. 19) 

(B .20) 


k 

The derivatives of J CD. with respect to the variances of the parameters 
involved in the second state CB.6) are the same as above with q^ replaced by 
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First Order Sensitivity 


3J*(1) 

3^(1) 


Using equations ($.13) and CB.14) we derive below 


aj*(i) 

30(1) 


. 2q ( FDe„(l)-EG0 9 (l)~FE0 (I)IGCG Cl)) 

= 2q. 0,(1)- 2 

98 (1) (CD-E ) 


ft t 

aj CD 
3e 2 (i) 


- 2q 1 {(FD0 ;1 Cl)-GE0 1 (D~FG0 3 (l)+G 2 e ? (D) (GD-E 2 ) 

- (F 2 D-2FGE+G 2 C) (6(1) D-9 0 (l) E) } / (CD-E 2 ) 2 

2 3 


SJ*C1) 

90 3 C1) 


= - 2q 1 {(F 2 0 3 (D-FE0 3L (l)~FG0 2 (l)+GC0 1 (l)) (CD-E 2 ) 
- ( F 2 D-2 FGE+G 2 C) ( C 0 3 ( 1) -E 0 2 (1) ) / ( CD-E 2 ) 2 


(B.21) 


(B . 22) 


(B . 23) 


JU ^ ■* 

The partials of J"(l) with respect to the parameters of the second state 
(B.6) are similar to the above with q^ replaced by q 2 » and the parameters 
®1’ ®2> ®3 ® 5 » Qg) respectively ones. 
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Second Order Sensitivity 


9 2 J*(1) 

30.C1) ae (i) 


^2 ^ = 2( ll ~ ' C2q 2 e^Cl)D - 4q 2 0 2 Cl) e 3 (l)E + 2q 2 6 2 (l) C) 


(B . 24) 


aejci). 


CCD-E ). 


3 J*(l) 


2 2 

(CD-E ) 


{(2q 2 0 1 (l)0 2 (l)D - 2q 2 0 1 (l)0 3 Cl)E - 2q 2 0 2 (l)F 


^ 3 2 j*(D 

30 1 (1) 90 3 (1) 


2 2 
(cd-eV 


a A A A 

+ 2 q p 2 (l)0 3 (l)G - 2GE qi + 2FD qi ) (CD-E z ) 


- (2q. 0 9 (1) FD - 2q 1 0-(l)GE - 2 qi 0,(l)FE 


+ 2q 1 0 3 (l)GC) (2 qi 0 2 (l)D - 2 q;L 0 3 (l)E) } 


{(2q 2 0 2 (l)0 3 (l)F - 2q 2 0 1 (l)0 2 (l)E - 2q 2 0 2 (l)G 


(B. 25) 


A a A A 

2FEq L + 2GCq 1 + 2qp i (l) 0 3 CD C) (CD-E ) 


(B.26) 


- (2q 1 0 2 (l) FD - 2 q;L 0 2 Cl)GE - 2 q;L 0 3 (l)FE + 2q 1 0 3 (l)GC) 


(C.2q 1 0 3 (l) - 2E q;L 0 2 (l))} 


J C ) = _ [{(2q 2 0 2 (l) D - 4q 2 0 (1)0 (1)G + 2q Q 2 ) (CD-E 2 ) 

a0 2 d-) (CD-E Z ) 4 11 1 1 J J- 


(F 2 D - 2FGE + G 2 C) (2q 1 D - 2q 2 0 2 (l))} (CD-E 2 ) 2 


- {(2q 1 0 1 (l) FD - 2q 1 0 1 (l)GE - 2 qi 0 3 (l)FG + 2 qi 0 2 (l)G 2 ) * (B.27) 


(CD-E 2 ) - (F 2 D - 2 FGE + G 2 C) (Zq^d) D "- 2q 1 0 3 (l)E)} 


• 2{CCD-E 2 )(2 qi 0 2 (l)D - 2q 1 0 3 (l)E)}] 
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9 * 

3 J Cl) 


ae 2 ci) 36 3 (i) 


[{(2qJe i Cl) 9 3 (1)F + 2q^e i (l)e 2 Cl)G - 2q^(l)E 

C CD-E ) 

- 2 qi FG) CCD-E 2 ) + (2 q;L 0 3 (l)F 2 - 2q.e-CD.FE 


- 2 q;L e 2 Cl)FG + 2q 1 0 1 (l) GC) (2 qi 0 2 (l)D ^q^CDE) 


- (2q 1 0 1 (l)FD -2q 1 0 1 (l)GE - 2q 1 0 3 C.l)FG + 2 qi 0 2 Cl)G" 


(2q 1 0 3 (l)C - 2q 1 0 2 (l)E) - (F 2 D - 2FGE + G 2 C)C2q 2 0 2 


2q x E) } CCD-E 2 ) 2 - {(2 qi 0 3 Cl)F 2 - 2q 1 0 1 Cl)FE 


-2q 1 0 2 (l) FG + 2q 1 0 1 (l) GC) (CD-E 2 ) - (F 2 D - 2FGE + g" 


(2q 1 0 3 (l) C - 2q 1 0 2 (l)E) } {2CCD-E ) (2 q;L 0 2 (l) D - 2q } 


2 * 

3 J Cl) 
ae 3 Ci) 


~-T HC2q F 2 - 4q 2 0 CDe Cl)F + 2q 2 0 2 Cl)C) CCD-E 2 ) 

CCD-E Z ) 1 XX 2 11 

- CF 2 D -2FGE + G 2 C)C2q 1 C - 2q 2 0 2 CD) } C.CD-E 2 ) 2 


- {C2q 1 0 3 C.l)F 2 - 2q 1 0 1 Cl)FE - 2q 1 0 2 Cl) FG + 2 q ^ 0^ Cl ) G C) C.CD-E 


CF 2 D - 2FGE + G 2 C) C2q 1 0 3 Cl)C - 2q 1 0 2 C.l)E) } {2 CCD-E 2 ) 


-2q 1 0 2 (l) E) }] 


) * 

Cl) © 3 C.1) 
CB.28) 
C) • 

6 3 C1)E}] 

CB.29) 

2 ) 

>3 Q-) C 


62 



2 

First and Second Order Sensitivities and — vr — 

3uC0 > 3 u 2 (0) 


2 

3P(1) 3r(l) 

Next we shall compute the terms and — ^ . Referring to 


3u 2 (0) 


CB.5-B. 11) we know that at any time the covariance is block diagonal. The 
parameters in (B.5) are uncorrelated with those in (B.6). The measurement 
matrix H(k) composed of the controls is also block diagonal. Thus we write 
the following. 


P(k) = 


(k) 0 


0 P (2) Ck) 


(B . 30) 


H(k) = 


H(k) 0 


(B. 31) 


Following any covariance update equation C3.8) of Appendix A we need 
H(1)P(0)H' (1) . Thus 


RCl)PCO)H’(l) = 


" HCD o 


"p (1) (0) 0 


H* 

O 

1 

0 HC1)_ 


0 P C2) CO). 


_0 H’C1)_ 


H(1)P^ (Q)H'(l) 


H(1)P (2) C0)H'(1) 


(B.32) 


Let us look into H(1)P^ (O)H' (1) further. 
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H(l)P^ ll (0)U’(l) 


- [ 1 


u x CO) 


u 2 C0) 


P l,].<°> P X,2<°> 

P 2,l (0 ' P 2,2^ 

P 3 >1 (0) P 3,2 t0) 


P 1,3 C0) 

P 2,3<°> 

P 3,3 (0) 


* 


1 

u 1 (0) 

u 2 (0) 


P l,l (0) + 2 u 1 (0)P 1 > 2 (0) + 2u 2 C0)P 1>3 (0) + 2u ] (0)u 2 (.0)P 2 3 (0) 
+ u 1 C0 > P 2> 2 C0) + u 2 (0) P 3,3 C0) 


(B. 33) 


Similarly, 

( 2 ) 

H C1)P V ' (O)H' Cl) - P 4>4 (0) + 2u 1 C0)P 4 5 C0) + 2u 2 (0) P 4 g (0) 


+ 2u 1 (0)u 2 (0)P 5>6 CO) + u 2 (0)? 5j5 C0) + u 2 CO)P 6 g(0) 


(B.34) 


Thus 


IH(1). P(0) H' Cl) + W] 


-1 


_i_ r & 0* 

l*B [_ 0 A _ 


(B. 35) 


where 


A = H(1)P (1) (0) H’(l) + W x 
B = H(1)P^(0) H* (!) + w 2 


(B . 36) 
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The innovation covariance in equation (3.8) of Appendix A is 


1 

A-B 


P (1) (0) 0 


0 P^(0) 


H'(l) 0 

0 HVCl) 


B 0 


0 A 


H(l) 0 
0 H(l) 


P^CO) 0 


0 P (2) (0) 


A*B 


BP (1) (0)H'(1)H(1)P (1) (0) 


AP^ 2) (O)H' CDHCDP^ 25 (0) 


(B . 3 7) 


With our standard notation P^ (0)H f (1)H(1)P^ (0) may be rewritten as 


P 1,2 C0 > P 1,3 C0 >' 


1 

... . 1 


' P 1 ( 1 C0) P 1.2 W > P l,3 C ° r 

P 2,1 C0> P 2,2 (0) P 2,3 C0 > 


Co) 

[1 u^CO) u 2 (0)] 

P 2,1 C0) P 2,2 C0 > P 2,3 W) 

_ P 3,lOM P 3 i2 (W P 3,3^ 0) J 


u 2 (0) 


. P 3,l t0) P 3,2 C0) P 3,3 W> . 


a 

8 

Y 


[a 8 y] 


a 


ay 

8a 

8 2 

By 


Y8 

2 

Y<x 

Y 


with 


“ = P 1,1 C °) + P l,2 C0)u l (0) + P 1 3 C0 )u 2 C0) 
6 = P 1>2 (0) + P 2>2 (0)u 1 (0) + P 2 j3 (0)u 2 C0) 
y = P lf3 C0) + P 2 3 (0)u 1 (0) + P 3 3 (0)u 2 (0) 


(B .38) 


(B.39) 
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Further introducing , 


6 = P 4,4^°) + P 4f5 (0)u 1 (0) + P 4>6 (0)u 2 C0) 
ip = P 4)5 (0) + P 5}5 C0) Ul (0) + P 5 6 (0)u 2 (0) 

* =P 4,6 C0 > + P 5,6<°>*1< 0 > + \6<°> a 2 C0 > 
we may write the innovation covariance as 



Let us define 

AD1 - T^(oT - 2P 1,2<°> + 2 u 2 (0)P 2j3 (0) + 2u 1 (0)P 2 2 C0) 

AD2 * sf(0T * 2P 1,3 (0) + 2u l ( ° )P 2,3 (0) + 2 u 2 C0)P 3i3 (0) 
3B 

BU1 - - 2P 4,5 (0) + 2p 2 (0)P 5>6 (0) + 2u 1 (0)P 5>5 (0) 

3B 

BU2 = lu^0> = 2P 4,6 C0) + 2 u 1 (0)P 5,6 C0) + 2u 2 (0)P 6,6 C0) 


(B.40) 


(B.41) 


(B . 42) 


(B. 43) 
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Thus we may next evaluate 


3P(D 

3u( 0) 


and 


9 2 P(1) 

8u 2 (0) 


These are given below. 


9p l,l (1> 

8 

/ 2 

\ 2aP 1 2 (0)A - a - AU1 


3u,(0) 

3^(0) 

l A 

) *— 7 



8 


\ 2aP l 3^ A " a 2 • AU2 


3u 2 C0) 

9u 2 (0) 

V A 

7~ ' 


3P 1,2<« 

8 

/ aB^ 

| . Pi, 2 C°)8A + o,P 2 2 CO)A - 

aBAUl 

9u 3 (0) 

3^(0) 

\ A i 



9Pl >2 CD 

8 

1 a& ■ 

{ P x 3 (0)BA + aP 2 3 C0)A - 

aS*AU2 

3u 2 C0) 

3u 2 (0) 

\ A , 



3Pl i3 U> 

8 

/_ ax' 

1 PjL 2 ( 0) yA + aPp 3 (0)A - 

ay AU1 

3^(0) 

3u 1 (0) 

\ A J 

A 2 ’ 


3Pl i3 U) 

8 


P 1 3 C0)yA + aP 3 3 (0)A - 

ay. AU2 

3u 2 (0) 

9u 2 (0) 


A 2 


3P 2j2 U) 

8 

i.l) 

2 BP 2 2 (0)A - B 2 -AU1 


3u 1 (0) 

3^(0) 

{ A ) 

' A 2 


»2,2<*> 

8 

I.l) 

2 BP 2 3 (0)A - B 2 • AU2 


9u 2 (0) 

3u 2 (0) 

\ A / 

A 2 



(B . 44) 


(B. 45) 


(B . 46) 


(B. 47) 


(B.48) 


(B. 49) 


(B.50) 


(B.51) 


67 



P 2 2 (0)yA + pp ? 3 (.0)A - 3y. 


j Lsx) * 

i ? (0) \ A / 


P 2 3 (0)yA P p 3 3<0)A - 3Y' • 


3 ^( 0 ) 


(-#) 


2yP 2 3 (0)A ~ Y * AU1 


3u 2 (0) 


('A") 


2yP 3 CO) A - Y ' AU2 

,2 ~ 
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9u^(0) 


-4 t2(P i >2 (0)A “ a2p 2 2 (0)) ^ " C2ct P 1 2 (0)A - a 2 . AU1) (2 A- AU1) ] (B.56) 

A * * 


3 ? u% a) 


SuTCoTfeo) = - 7 [2(P 1,2 (0)P 1,3 (0)A + - «AU2P 1(2 C0) 


- a 2 P 2 3 (0))A 2 - (2aP 1 3 (0)A - a 2 AU2) (2A-AU1) 3 


(B.57) 


5u 2 (0) 


\ [2(P 2 3 (0)A - a 2 P 3 3 (0))A 2 - (20^ 3 C0)A - a 2 - AU2) C2A- AU2) ] (B.58) 


3 P l,2 (1 > 

3u 2 (0) 


- - \ [2(P 1, 2 (0) P 2 ,2 (0)A ~ «3P 2 2 C0))A 2 - CP X 2 C0)BA + aP 2 2 (0 )A (b.59) 

A ^ ^ 

- aBAUl) (2A- AU1) ] 


c P l,2 (1) 


3^W&u 2 ~ (0) ' " ' 7 [(P 1,3 C0)P 2,2 (0)A + P lj3 (0)6AU1 + P 1,2 (0)P 2,3 (0)A+ aP 2 ,3 (0)AU1 


- P x 2 (0)BAU2 - aP 2 2 ( 0) AU2 - 2a£P 2 3 C0))A 


C.B.60) 


- C? x 3 (0)BA + aP 2 3 (0)A - a BAU2 ) (2 A • AU1 ) ] 


3 P 1 2 (1) • 1 2 

* = - ±r- f 2 C p i ,(0)P 9 o(0) A - aBP. ,{.0))A 4 - CP, 9 C0) BA + aP (0)A 

3u 2 (0) A i,J ' J »' i,:S Z ’ J 

- aBAU2) C2A-AU2) ] 


CB.61) 


3 P 1 3 (1) 1 2 

5 4 [2(P 1>2 (0)P 2 j 3 (0)A - ayP (0))A 

A 


3u 2 (0) 


- ^ 2 (0)yA + aP 2 3 (0)A - ayAUl) (2A-AU1) ] 


(B . 62) 
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^ P 1 3^^ 1 

— *- = - —r t(p o(0)P (0)A + P _ ( 0) yAUl 4- P 9 (0)P„ _(0)A 

3 Ul (0)3u 2 (0) A X ’ l ' ±,J i » / 


+ cxP 3 3 (0)AU1 - P x 2 (0)yAU 2 - aP 2 3 


(0) AU2 


(B . 63) 


- 2ayP 2 3 (0))A' i - (P ]L 3 (0)yA + aP 3 3 (0)A - ayAU2) . (2A- AU1) ] 


3 P 1,3 CI > 1 


3u 2 (0) 


4 t 2 CP l,3 C 0 )? 3,3 (0)A " °YP 3 j 3 ( 0 ))A 2 - (P L 3 C0)yA + aP 3 3 (0)A 


(B. 64) 


- ayAU2) (2A-AU2)] 


aP 2,2 (1) 

3u 2 C0) 


= - K- [2(P 2 „ (.0) A — 6 2 *P 0 o ( 0))A 2 - C26P 0 - ( 0) A - B 2 AU1) C2A- AUl) ] (B.65) 


• 2,2 


2,2 


2,2 


3 P 2,2 (1) 


3u^(0) 3 u 2 (0) A 


= - [2(P 2}3 C0)P 2>2 (.0)A + 6P 2j3 C0)AU1 - 3AU2P 2>2 


CO) 


(B. 66 ) 


- 6 2 P 2 3 (0))A 2 - C26P 2 3 (0)A - 6 2 AU2) (2A-AU1)] 


9 P 2,2 (1) 

3u 2 (0) 


= - [2 (P 2 3 (0)A - 3 2 P 3 3 (0))A 2 - (23P 2 3 «))A - 3 2 AU2) C2A* AU2) ] (B.67) 


3 P 2 3 C1) 1 2 

2 = [ 2 ( p 2 t 2 ( 0 )p 2 , 3 (0)A " 3 yP 2 , 2 C0))A - (p 2 2 C 0 ) y a+ bp 2 3 (o)a 


3u 2 CO) 


(B. 68) 


ByAUl) (2A-AU1)) 


3 P 2 > 3 (1). 

3 u^( 0 ) 3u 2 (0) 


-4 [P 2 3 (0)A + P 2 3 (0)yAUl + P 2 2 (0)P 3 3 C0)A + BP 3 3 C0)AU1 


P 2 2 ( 0) yAU2 - BP 2 3 <0)AU2 - By 2P 2 3 (0))A 


CB.69) 


- (P 2 3 C0)yA + BP 3 3 (0)A - ByAU2) (2A-AU1) ] 
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9 P 2,3 (1) 1 


3u 2 (0) 


4 I2(P 2,3 (0)P 3,3 (0)A “ 3yP 3 3<Q))A Z - (P 2 3 (0 )yA+ BP 3 3 (0)A (B.70) 

- ByAU2) (2A-AU2) ] 


3 P 3 3 C1) 1 2 

9 _ 4- To 


3u 2 (0) 


4 t2(P 2,3 (0)A " Y 2p 2> 2 (0))A2 ~ C 2 Y P 2 3 (0)A - y 2 AUl) (2A- AU1) ] (B.71) 


3 P 3,3 (1) 
° U 1 (0) 3u 2 (0) 


T t2 ^ P 2 > 3 CO)P 3,3 CO)A + yP 3 3<°) au1 - Y p 2 3 (0)AU2 
- Y 2 P 2 3 (0))A 2 - (2yP 3 3 (0)A - y 2 AU2)(2A*AUl) ] 


(B.72) 


3 P 3,3 (1) 
3u 2 C0) 


^4 [2(P 2 >3 (0)A - y 2 P 3j3 (0))A 2 - (2yP 3 3 (0)A - y 2 AU2) • (2A-AU2) ] (B.73) 


The partials of the covariances associated with the parameters of (B.6) are 
similar to the above with a, Q, y. A, AUl, AU2 replaced respectively by 6, \p, <p, 
B, BUI, BU2 and the appropriate covariances of the parameters of (B.6). 
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LONGITUDINAL HUB SHEAR STATE XI 



Fig. 1 Longitudinal hub cosine state 

xi, vs the control inputs 63 c and 03 s from the 
identified third-order nonlinear model obtained 
from G40Q simulation data (120 knots) 
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Fig* - - Longitudinal hub sine state 

x 2 vs the control inputs 63 c and 835 from the 
identified third-order nonlinear model obtained 
from G400 simulation data (120 knots) 
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Comparison of the cautious and dual controllers 
performances on the 30% varying parameter plant 
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A Dual, 3=1 
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Comparison of the cautious and dual controllers 
performances on the 30% varying parameter plant 
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Fig. 12 Time history of 9^, 9^, 

9 3 — , using the cautious controller 
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Fig, 16 Comparison of the performance of the cautious 
controller with that of no control (second 
method of initialization) (Run 1) 



Fig. 17 Control u-^ with the cautious controller 

(second method of initialization) (Run 1) 
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(second method of initialization) (Run 1) 
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Vibration contribution from 

:he cosine component, using the cautious controller 
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Comparison of the ratio of the eigenvalues of 
the true and estimated parameter transfer matrices 
for the cautious controller 



o No Control 
A 6 
+ 8 



I *.00 J0.OO 

MME srfcp 


tB.yO 32 ■ vC 


Fig. 29 Comparison of no control, cautious and the first 
order dual controller's performances on the time 
varying parameter case (30%) Run 1 
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g. 32 Comparison of no control, cautious and 
order dual controller's performances oi 
varying parameter case C30%) Run 4 


o No Cont 
A 3=0 
+ 3=1 


-2 00 i*> 00 JO. CO 2* 

MM?: STEP 


j, 33 Comparison of no control, cautious and 
order dual controller’s performances or 
varying parameter case (30%) Run 5 
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Fig. 34 Comparison of no control* cautious and the first 
order dual controller’s performances on the time 
varying parameter case (30%) Run 6 
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Fig. 35 Comparison of no control, cautious and the first 
order dual controller's performances on the time 
varying parameter case (30%) Run 7 
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Fig. 36 Comparison of no control, cautious and the first 
order dual controller's performances on the time 
varying parameter case (30%) Run 8 
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Fig. 37 Comparison of no control, cautious and the first 
order dual controller’s performances on the time 
varying parameter case (30%) Run 9 
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Fig. 38 Comparison of no control, cautious and the first 
order dual controller's performances on the time 
varying parameter case (30%) Run 10 
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Fig. 39 Comparison of no control, cautious and the first 
order dual controller’s performances on the time 
varying parameter case (30%) Run 11 
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Fig. 40 Comparison of no control, cautious and the first 
order dual controller’s performances on the time 
varying parameter case (30%) Run 12 
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Fig. 41 Comparison of no control, cautious and the first 
order dual controller's performances on the time 
varying parameter case (30%) Run 13 
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42 Comparison of the no control, cautious and 
first order dual controller's performances 
the time varying parameter case (30%) Run 1 
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43 Comparison of the no control, cautious and t 
first order dual controller's performances o 
the time varying parameter case (30%) Run 15 



k Comparison of the no control, cautious and t 
first order dual controller’s performances o 
the time varying parameter case (30%) Run 16 
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5 Comparison of the no control, cautious and tl 
first order dual controller's performances oi 
the time varying parameter case (30%) Run 17 
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Fig. 46 Comparison of the no control, cautious and the 
first order dual controller's performances on 
the time varying parameter case (30%) Run 18 
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Fig. 47 Comparison of the no control, cautious and the 
first order dual controller's performances on 
the time varying parameter case (30%) Run 19 
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Fig. 48 Comparison of the no control, cautious and the 
first order dual controller * s performances on 
the time varying parameter case (30%) Run 20 
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Lg. 49 Vibration contribution from the cosine 

component, using the dual controller (second method 
of initialization) (Run 1) 
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Fig. 56 Time history of 0^, 0^, 

0 4 + , using the dual controlle 


o 0, 






Cost l x 1000 £ Ratio of Eigenvalue 


RUN 1 


o True 
A. Estimated 


XA. 


4.QC 8-00 


»?-00 16- CO 20.00 24-00 

TIME STEP 


28. 0C 32-00 35. CO 


60 Comparison of the ratio of the eigenvalues of 

the true and estimated parameter transfer matrices 
for the dual controller 
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Fig* 61 Vib.ration contribution from 

the cosine component using the cautipus controller 
(second method of initialization. Run 2) 
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Fig. 63 Time history of control & used 
by the cautious controller 
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Fig. 66 Vibration contribution from the. 

sine component using the dual controller (Run 2 ) 



Fig- 67 Time history of control 1 used 
by the dual controller 







Determinant 3 x 1000 to Ratio of Eigenvalues 



TIME STEP 
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Fig. 71 Comparison of the determinants of 
and the estimated parameter transf< 
for the dual controller 
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Comparison of the ratio of the eigenvalues of 
the true and the estimated parameter transfer 
matrices for the cautious controller 
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Comparison of the determinants of the true 
and the estimated parameter transfer matrices 
for the dual controller 
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Fig. 104 Time history of optimal control 2 for Run 
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Fig. 105 Time history variation of the total cost 

with different switches on the control weights 
for Run 2, (Q=diag(l, 1) ) 
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108 Time history the variation of t 
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for Run 2. (R=diag(0,0)) 
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cont ri but ion w ith di f f eren t s 
the state weights f Q T for Run 
(R=diag(0,0) ) 
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Fig- Time history variation of the sine 

contribution with different switches on 
the state weights ’Q r for Run 2. (R=diagC-01, . 01) ) 
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Fig. Ill Time, history the variation of the total cost 

with different switches on the state weights ’Q* 
for Run 2. (R=diag(.01, .01) ) 
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Fig. 112 Time history the variation of the cosine 
contribution with different switches on 
the state weights 1 Q f for Run 2. (R=diag( * 01, * 01) ) 
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Fig. 113 Time history variation of the sine 

contribution with different switches on the 
state weights f Q f for Run 2. (R=diagC. 01, . 01) ) 
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Fig. 114 Effect of the initial covariance 

(.first method of initialization) on the cautious 
and the dual controllers 
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Fig. 115 Effect of the initial covariance 

(first method of initialization) on the cautious 
and the dual controllers 
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Fig. 116 Effect of the initial covariance 

(first method of initialization) on the cautious 
and the dual controllers 
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Fig. 117 Effect of the initial covariance 

(first method of initialization) on the cautious 
and the dual controllers 
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Fig. 118 Time h.£&toxy convergence of cost for the 

global linear adaptive cautious controller showing the 
divergence points when the control values are in a very 
nonlinear region. Q=diag(10~5 > 5xl0~ 8 ) ; R=diag(10^^, 10“^) 
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o 



Fig. 119 Time history convergence of Control 1 for 

the global linear adaptive cautious controller showing the 
divergence points when the control values are in a very 
nonlinear region. Q=diag(10~5 > 5xl0~8) ; R=diag(10~^ > lO - ^) 
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Fig. 120 Time History convergence of Control 2 for the 

global linear adaptive cautious controller showing the divergence 
points when the control values are in a very nonlinear region. 

Q=diagCL0-5, 5x10-8) . R=diagCl0-4, 10-4) 
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Fig. 121 Time history convergence of cost for the 

global linear adaptive cautious controller showing the 
divergence points when the control values are in a very 
nonlinear region. Q=diag(l,l) ; R~diag(0,0) 
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Fig. 122 Time history convergence of Control 1 

for the global linear adaptive cautious controller showing 
the divergence points when the control values are in a very 
nonlinear region. Q=diag(l,l) ; R=diag(0,0) 



Fig. 123 Time, history convergence of Control 2 

for the global linear adpative cautious controller showing 
the divergence points when the control values are in a 
very nonlinear region. Q=diag(l,l) • R=diag(0,0) 
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V=. 1 . B= . 05. P=1 .0 



Fig. 124 Comparison of the costs using the cautious, the first 

order dual and the new dual (Time varying parameter case. 
Run 2 from 100 Monte Carlo Runs) 
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Fig . 125 


Comparison of the controls using the cautious, the 
first order dual and the new dual (Time varying 
parameter case. Run 2 from 100 Monte Carlo Runs) 
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Fig. 130 Comparison of the cost using the cautious, the first order 
dual and the new dual (Time varying parameter case: 

Bun 21 from 100 Monte Carlo Runs) 
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Fig. 131 Time- history of the. true parameter for 

Run 21 from 100 Monte Carlo Runs (Time varying case) 
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Fig. 134 Time history of the control using the cautious, 

dual and the new dual solutions (Constant parameter 
case: Run 26 from 100 Monte Carlo Runs) 


•W=.l, y = . 0, B=.Q5. P=1 . o 



Fig* 135 Time history of the cost using the cautious, dual 

and the new dual solutions (Constant parameter case: 
Run 44 from 100 Monte Carlo Runs) 
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Figure 138. Time history of cost comparing 
the new dual, dual of Reference 1 and 2, 
and the cautious controller (Time varying 
parameter case: Run No. 7 fora 100 Monte 

Carlo Runs) 


Figure 140. Time history of cost 
comparing the new dual, dual of 
Reference 1 and 2, and the cautious 
controller (Time varying parameter 
case: Run No. 14 from 100 Monte 
Carlo Runs) 
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Figure 139. Time history of parameter for 
Run No. 7 from the 100 Monte Carlo Runs 
(Time Varying Case) 
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TIME STEP 


Figure 141. Time history of parameter 
for Run No. 14 from the 100 Monte Carlo 
Runs (Time Varying Case) 
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TIME STEP 


Figure ‘142. Time history of cost comparing 
the new dual, dual of Reference 1 and 2, 
and the cautious controller (Constant 
parameter case: Run No. 26 from 100 Monte 
Dario Runs) « - 


Figure 143. Time history of cost 
comparing the new dual, dual of 
Reference 1 and 2, and the cautious 
controller (Constant parameter case: 
Run No. 80 from 100 Monte Carlo Runs) 


146 




NO 

CAUTIOUS 

FIRST ORDER 
DUAL 


CONTROL 

CONTROL 

CONTROL 

COLUMN 

Cl 

C2 

C 3 

COUNT 

100 

100 

100 

Rnw 

1 

29533. 

2021 5.7 

15764.2 

7 

433 SI . 

12057.3 

1257,1.0 

3 

13404. 

13034.1 

9945.3 

4 

503 53 . 

20402.7 

24502.0 

5 

19526. 

13650.5 

12523.1 

6 

98931 . 

24216.8 

18334. 9 

7 

1 4 5 6 ? 5 • 

17844.4 

1355S.6 

8 

. 133 33. 

4515.5 

4440.5 

9 

128733. 

6 3888. 3 

58591.4 

10 

73020. 

43092. 1 

34656.7 

1 1 

28091. 

3360.7 

3230.2 

12 

88536. 

23937.9 

• 2 3491.6 

13 

15391. 

2454.7 

2392.5 

14 

22420. 

8319.9 

P380.3 

1 5 

68888. 

36004.2 

36422.2 

16 

49979. 

9125.8 

9718.3 

17 

403 32 . 

29413.5 

31975.5 

18 

28350. 

17064.0 

12511.5 

19 

71 219. 

42689.5 

3641 2.3 

20 

28107. 

5566.4 

6045.1 

21 

53340. 

54353.8 

56635.1 

22 

13255. 

4186.4 

4224.0 

23 

1 36934. 

46734.9 

37071.6 

24 

63349. 

4702 9.4 

47367.5 

25 

26763 . 

2731.2 

2316.2 

26 

10338. 

1 294 1 . 2 

12575.2 

2 7 

49 111 . 

2882 5.9 

31378.5 

28 

15025. 

16803.9 

17190.7 

29 

39763. 

1 034 5. 1 

10571. 0 

3 0 

24362. 

347 1 . 2 

3441.4 

31 

781 34. 

54 L 1.6 

5368.2 

3? 

99955. 

26234.7 

21871.2 

33 

181477. 

3419 l. 8 

40623.4 

3 4 

19294. 

1 668 8.0 

14996.0 

39 

31517. 

8046.2- 

5585.5 

3 6 

22659. 

3155.3 

3104.3 

37 

40703. 

753 7 . 5 

6238.2 

38 

34430. 

7443.4 

611 8. 6 

39 

12744. 

1119 7.6 

7963.5 

40 

51 355. 

10246.9 

10391.1 

41 

29747. 

8306.3 

6923.6 

42 

5 09 ?cj. 

1778 l, 0 

18006.5 

43 

35569. 

5255. 3 

5336. 4 

44 

17551 . 

2103.3 

2152.5 

4 5 

7542. 

3194.2 

3312. 1 

4 6 

Q9625. 

41247.9 

46671.4 

47 

560 21 . 

1174?. 8 

16181 . 3 

4 8 

1 64647. 

39493.9 

40825. 1 


Table 1. - Cost Values for the 1Q.Q Monte-Carlo Runs For 30% 

Process Noise (No Control, Cautious Control, First 
Order Dual Control) 
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NO 

CAUTIOUS 

FIRST ORDER 


CONTROL 

CONTROL 

DUAL CONTROL 

49 

30097. 

9734.6 

7329.8 

5 0 

1O307. 

9C3 1.9 

10811.2 

5 l 

53184. 

13510.8 

13252.4 

5? 

50729. 

4210.2 

4203.9 

53 

530 32. • 

37536.4 

33789.5 

84 

123404. 

5531.1 

5453.6 

55 

24251. 

4851.5 

4915.2 

56 

23350. 

5755.2 

6763. 1 

67 

92454. 

29068.3 

29249.0 

5 9 

71128. 

2722 7. 1 

29648.5 

59 

12923. 

2838.6 

2737.8 

60 

501 50. 

1919 4.0 

17414.1 

61 

61531. 

5055.7 

5100.6 

62 

39167. 

22284.4 

14690.5 

63 

123726. 

59607.8 

63676.0 

64 

49263. 

17388.0 

16184.6 

65 

89602 . 

13362.7 

11733.3 

66 

74954. 

54910.7 

50507.4 

67 

32930. 

8820.3 

7394.6 

63 

46325. 

10346.8 

9781.0 

69 

42513. 

460 5.5 

4531. 0 

79 

57433. 

1297 7.0 

6851.2 

71 

49134. 

27630.5 

29720.5 

72 

94379. 

47743.1 

43412.0 

73 

39835. 

21408.3 

22353.1 

74 

105392. 

5279.1 

5417.2 

75 

42173. 

4391 1.5 

22868.5 

76 

10164. 

3052.9 

3067.1 

77 

14932. 

491 0.6 

4917.6 

73 

64611. 

34968.2 

37813.5 

79 

9705. 

3012.9 

3228. 5 

80 

15149. 

11303.1 

10163.5 

81 

16436. 

2971.0 

2827.1 

82 

38390. 

21377.7 

31838.4 

R3 

10240. 

4177.8 

4126. 1 

84 

26971 . 

8805.4 

8825.1 

95 

30797. 

5239.7 

4738.5 

86 

62 79. 

6468.3 

4024.6 

87 

22690. 

11025.6 

11231.4 

88 

52721. 

12340.1 

9472.4 

89 

21636. 

99? 5.7 

8603.5 

90 

62 7 64. 

3277 8.4 

31479. 5 

91 

192^4. 

6233.4 

6445.6 

99 

23055. 

6020.5 

5298.4 

93 

98388. 

72592.9 

68177.6 

94 

73330. 

2239 3.3 

17316. 1 

O P 

99183. 

15872.6 

17329. 3 

96 

10049. 

6202.7 

7427.8 

07 

4^507. 

23657.6 

24194.7 

o p 

81738. 

25497.7 

25347.4 

99 

43204. 

3935.7 

3984.1 

10 0 

4 62 5?. 

1814 1. 4 

15749.8 


Aver No Control = 50531. 

Aver Cautious Control = 18051. 
Aver Dual Control = 17141. 


Table 1. CCont.) 
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Table 2. 


ORIGINAL PAGE IS 
OF POOR QUALITY 


CAUTIOUS FIRST ORDER SECOND ORDER 

DUAL DUAL 


m i i im.m 

r i 

C2 

H. 

COUNT ' 

130 

190 

100 

ROW 




1 

0.293600 

3.794 10Q 

9.287000 

7 

0 . 133 000 

0. 1 76 800 

0. 170000 

3 

0 . 400 200 

9.347600 

0.297300 

4 

0 . 573600 

3. 577100 

0. 6.6 270O 

3 

0.333700 

0.343 590 

0. 376900 

6 

0 . 51 1 700 

3.491 973 

0.453330 

7 

3 .477500 

7. 413 890 

0.41 4330 

3 

3.533200 

0.613600 

0.561700 

9 

0.423 200 

3.425900 

0.41 5 930 

10 

0 .53 6900 

9. 505600 

0. 527000 

1 1 

0 . 71 0600 

3. 708 100 

0 . 752400 

1 2 

0.353900 

0.347400 

0. 340 200 

1 7 

0.742500 

9.764 109 

0. 701800 

i 4 

0.315 500 

3.312 330 

0.798500 

1 5 

0.7714-00 

3 . 403 1 00 

0.406100 

16 

D. 369 600 

0.367100 

0.773000 

1 7 

0.770290 

9.773700 

3. 759070 

1 3 

0 . 442 30 0 

0. 431 200 

0.418 40 0 

1 9 

0.741 700 

3.722 300 

0. 76 6000 

7 3 

0 .523500 

0. 529 800 

0. 49 2 20 0 

2 1 

n ,624300 

0. 407 5H0 

0.561500 

22 

0 .277300 

7.73 7 700 

3.227110 

7 7 

0 . 57 6 700 

9. 581000 

0. 583800 

24 

0.134800 

0. 129200 

0. 125700 

2 5 

0 . 497 500 

0. 7Q>3 900 

0 . 42 3 70 0 

26 

0. 160400 

0. 135000 

0. 111100 

2 7 

0.573600 

0. 569300 

0.553800 

2 8 

0.175400 

0.175100 

0. 18 9 200 

2 9 

0.724000 

0. 740900 

0. 740900 

30 

0.286609 

1.292800 

0.298433 

3 1 

0 . 605300 

0. 613 400 

0. 524800 

32 

0 . 565 300 

0.557200 

0.523900 

73 

0.439300 

0 .439 700 

0 . 45 1 99 0 

34 

0.729400 

0. 703200 

0. 678100 

3 3 

0.60139'} 

0.592400 

0. 566990 

36 

0.270700 

3. 287200 

0. 208100 

37 

0.327100 

0.779000 

0. 33 0 10 0 

3 8 

0.690900 

3.683 700 

0.674530 

39 

0 .489300 

0. 463000 

0.436900 

4 O' 

0.373900 

0.367500 

0.341100 

41 

0 . 167 800 

3.168800 

0. 18 449 0 

42 

0.616100 

0.622300 

0.586700 

43 

0.616790 

9.62960C 

0.657490 

4 4 

0.471000 

0. 407 1.0 0 

0.458830 

43 

0.724200 

;j . 8oo ooo 

0.65 1900 

46 

0.583299 

0. 588790 

3.567593 

47 

0.210400 

0. 209400 

0.21 5000 

43 

0.737 300 

0.681200 

0.627 3,90 

49 

3.51 7800 

0. 477 000 

3. 470000 

5 0 

0.480200 

0.519800 

0. 453900 

5 l 

0 . 503 1 00 

3.506 l 30 

0.456299 

3 9 

3 . 640000 

3. 63 4 69 0 

0.677600 

57 

9.539200 

3. 573 400 

0.56 8 800 

5 4 - 

7 . 59 4 3 09 

3.615933 

9.600010 

59 

0 . 303900 

0. 305 300 

0. 30 1800 


- Cost Values for the 100 Monte-Carlo Runs for the time 
varying scalar model, using the cautious, first order 
dual and second order dual controllers (b(0) = .05, 
P b (0) = 1, V = .1, c = 1, W = .01, No Control Cost = 1) 
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CAUTIOUS 

FIRST ORDER 

SECOND ORDER 


DUAL 

DUAL 


•5 h 

9 . 51 1 ' u 't>l 

1 ) .4 t J 1UO 


.462500 

5 7 

0 .673000 

3 . 577400 

0 

. *88400 

* 0 

9 . 360700 

7.376600 

0 

.377000 

59 

3 . 347 60 j 

7.345 300 

8 

. 34 800 9 

6 9 

0 . 63 3 700 

3. 54 6 30O 

9 

. 56 2 60.1 

61 

0.203900 

0. 297 300 

9 

. 314700 

6 2 

) .440036 

'.445 770 

.) 

. 42 779 9 

63 

0 . 460 1 00 

0. 451 800 

0 

.436700 

64 

0 . 608000 

0.571900 

0 

. 548400 

66 

0 .273900 

9.382700 

0. 23 8 50 0 

6 6 

0 . 610700 

0.646600 

0 

.641109 

6 7 

0 , 47 4 70 0 

0.427 50 0 

0 

.440107 

6 8 

0 .473600 

1. 488 luO 

0 

. 496800 

6 9 

0 . 197900 

0.193200 

0 

• 19 &300 

m 

9.45949 9 

0.497600 

0 

. 396290 

71 

0.271 500 

0.2 39 COO 

0 

.230290 

72 

0.512100 

0.506500 

0 

.497200 

73 

0 .408200 

0.416000 

0 

. 33 560 O 

74 

0.565600 

0. 560 39Q 

0 

. 555700 

7 6 

■ 0.793100 

0.793 933 

0 

. 749939 

76 

0 . 700300 

9 . 709 900 

9 

. 741 59 0 

77 

9 . 589000 

0.509000 

0 

.471 400 

78 

9.794 7*00 

9.296 900 

9 

.318797 

76 

0.326400 

0. 310200 

0 

. 306900 

o 0 

0 .737700 

0. 66 8 900 

0 

.67320 0 

8 l 

0.473 40 0 

3 . 47.7000 

0 

. 416100 

8 2 

0.314700 

0.319700 

0 

. 326109 

8 3 

0.915430 

3.763930 

n 

.751200 

3 4 

0 .451600 

0. 451 10 0 

6 

. 537700 

85 

0.481900 

•9 . 503 000 

0 

.469500 

86 

0.196300 

3.197193 

0 

. 19 2 70 9 

3 7 

0 .449600 

0. 417 600 

0 

. 402200 

3 3 

0. 38310Q 

0.328100 

0 

. 348700 

89 

0 .493900 

,7.488 ion 

0 

. 468900 

00 

0 . 708500 

0.695700 

0 

. 60 4 3(3 0 

9 1 

9 .43339 0 

3.605 430 

0 

. 543309 

02 

0 .423000 

o. 431 400 

0 

. 42 6 400 

03 

0.6539QO 

0.664800 

0 

. 648400 

0 4 

0 .626190 

3 . 484 900 

0 

. 47 4 500 

o 5 

0 .538800 

0. 528 9QO 

0 

. 516100 

96 

3 .4031 3Q 

0.475930 

0 

.452170 

97 

0 .575900 

3. 518 609 

3 

. 49 4600 

98 

9.319200 

0.317400 

0 

.382090 

99 

9.263939 

7.269230 

0 

. 210609 

00 

0.427100 

0.379800 

0 

.328900 


Table 2. CCont.) 


Average Cautious = .47520 
Average First Order Dual = .46988 
Average Second Order Dual = . 45814 



Table 3. 


ORIGINAL PAGE IS 
OF POOR QUALITY 


CAUTIOUS FIRST ORDER SECOND ORDER 

DUAL DUAL 


column 

C 1 

C2 

C 7 

COUNT 

ROM 

100 

too 

109 

1 

0,37730 

O. 9705 O 

0.314000 

2 

0.24700 

0.24900 

0.179600 


0.3 8 00,;) 

0.37370 

0 . 45 8 49 O 

4 

0.59 30 0 

0.76590 

0.680600 

5 

0.51 770 

0.53640 

0.494609 

6 

0.63830 

0. 87 830 

0. 511 69 0 

7 

0.73070 

0.77080 

0.475400 

8 

0.53160 

0.57353 

C. 530000 

0 

0.49980 

0.49640 

0.434800 

1 o 

0.91900 

0. 89 96 0 

0.620300 

1 1 

C. 92020 

0.88 570 

0.613200 

l 2 

0.40770 

0.34300 

0.346700 

1 3 

0,79050 

0.95520 

0 . 80 5990 

1 4 

0.87560 

0. 36210 

0, 9 09 50 0 

1 5 

0.43750 

0,60 39 0 

0.425090 

16 

0.3 9 72 0 

0.36 73 0 

3.331 590 

1 7 

0.07439 

0.94370 

0.849800 

1 9 

0.67040 

0.55430 

0. 6?Q9°0 

1 9 

0.93 940 

0.90890 

°. 788800 

2 0 

0.47070 

0.63930 

0. 64630° 

21 

0. 77 580 

0.72450 

0 . 79 2 70 0 

7 7 

0.42810 

0. 72 980 

0 . 1 9 5 6U 0 

2 3 

0.76 76 0 

0.70550 

0.670090 

2 4 

9.27040 

0.214)0 

0.13 8790 

2 5 

0.42 100 

0.45810 

0.476600 

26 

0.41910 

0.21630 

0.200200 

27 

0.65590 

0.74280 

0. 585800 

28 

0.1762 0 

0.17340 

0.170900 

79 

0.87 66 0 

0.82 53 C 

0.631639 

30 

0.29970 

0.29500 

0. 296100 

3 1 

0.92650 

0.93 77 0 

0. 69 3 500 

32 

0.49660 

9,48 5)0 

). 514200 

33 

0,7069 0 

0.74210 

0. 587700 

34 

0.89090 

0.90 81 0 

0.711600 

3 6 

0.93930 

0.72410 

0. 6 10500 

36 

0.37420 

0.56320 

0.27030° 

37 

0.40930 

3.42 160 

9.39460° 

38 

0.74320 

0.72170 

0 . 683700 

3 9 

0.62200 

0. 53430 

0.47090° 

4 0 

O.60O3O 

0.57520 

0.39750° 

41 

0.24620 

0. 2415 0 

0.172600 

4 2 

0.78480 

0.76770 

0. " 7 9420O 

43 

0.72470 

0.70390 

0.678100 

44 

0.49 32 0 

0,47470 

0 . 48 470° 

45 

0. 77450 

0.74520 

0.641730 

46 

0.63010 

0, 69000 

0.628300 

4 7 

0.34190 

0.30030 

0, 23.2 000 

4 8 

1. 74 0.7 0 

0.79 820 

0. 719700 

49 

0.77530 

0.63850 

0. 61500° 

5 0 

0.72100 

0.75330 

0. 65 7(50 0 

c 1 

0.79230 

0.66 62 0 

0, 408500 

5 2 

0.75490 

0.72 320 

0. 71 160° 

53 

°. 98688 

0.91 870 

0.6573'*° 

5 4 

0. 86690 

0.66760 

0.59010° 

Cost Values 

for the 100 Monte-Carlo 

Runs for the time 

varying scalar model. 

using the cautious, first order 

dual and second order 

dual controllers (b(0) = .05, 

P b CQ) = 1, 

V = .1, c - 

= 1, W = .1, No 

Control Cost = 1) 


151 



CAUTIOUS 

FIRST ORDER 

SECOND ORDER 

' 

DUAL 

DUAL 


5 5 

0.47710 

0.44530 

0. 

478000 

5 6 

9.5566 0 

0.54210 

0. 

48 6 39 0 

5~r 

0.87470 

0. 88 87 0 

9. 

706500 

5 8 

0.43060 

0.43240 

0. 

735390 

5 9 

0, 44-26 0 

o. 37390 

0. 

312309 

SO 

0. 56 830 

0.63130 

n. 

650890 

61 

0.46280 

9*74690 

* *■ • 

771133 

6? 

0.58940 

0.59 580 

0. 

426100 

S3 

0.55230 

0.53890 

0. 

435730 

64 

0.66680 

0.61 32 D 

0, 

72 4 590 

S 5 

0,40000 

0.43910 

0. 

393200 

6 6 

0.82600 

3 . 7 9 93 3 

0. 

32 ICO 3 

61 

0.54170 

0. 43290 

0. 

412690 

68 

0,50030 

0.50240 

0. 

430090 

6 9 

O.8504.3 

9.59 2 7’7 

9. 

734299 

70 • 

6.83090 

0.84170 

0. 

420200 

71 

0.28470 

0. 2918 0 

0. 

706700 

12 

0.77530 

0.75140 

0. 

552200 

73 

0.45820 

0.52040 

9, 

437 500 

74 

0.741 78 

0 .7398 C 

' • 

637030 

75 

0.8731 0 

0 , 3 9 910 

n • 

792490 

76 

0.75730 

0.74240 

0. 

3 5 9 10 0 

77 

0. 84 69 0 

9.93 533 

3 . 

451990 

78 

0.32 95A 

6.33079 

0. 

328900 

79 

0.64820 

9 . 46 360 

0. 

456530 

8 9 

0.94980 

0.90939 

0. 

329100 

3 1 

0.46220 

0.57210 

0. 

485300 

8 2 

0.3122-3 

0.31463 

J • 

30 6630 

83 

1 .04020 

1.02040 

■ 0. 

79 0 700 

84 

0.66560 

0.63370 

0. 

629800 

85 

0.61 360 

3.63 51 3 

0. 

663330 

86 

0.25040 

0.98530 

0. 

212 59 0 

87 

0.46860 

0.4731 0 

0. 

450300 

3 8 

0.44730 

0.52830 

u. 

466900 

8 9 

n. 74990 

9.75400 

0. 

43 5100 

90 

0.83 94 0 

,o ,75 son 


73 l 3.) 9 

9 1 

0 • 6 K 1 5 0 

0. 73 070 

0 . 

588390 

02 

0.73680 

0. 33110 

n. 

538000 

9 3 

0 . .8 8 7 ,q 7 

9.71730 

0. 

63 -3 93 0 

9 4 

0.77620 

0.4® 63 0 

0. 

511200 

95 

0.32630 

0.71 890 

0, 

599700 

96 

1.04370 

0.55 310 

0 • 

541.890 

97 

0.70900 

0.79760 

0. 

635209 

0 8 

0.6 001 3 

3 .62 64:3 

3. 

366430 

99 

0.75800 

9.73070 

3. 

270 79 0 

0 0 

0.47860 

0.49490 

0 . 

43 0.200 


Average Cautious = .62293 
Average First Order Dual = .60787 
Average Second Order Dual = .51389 

Table 3. ( Cont . ) 
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CAUTIOUS 



FIRST ORDER SECOND ORDER 
DUAL DUAL 


COLUMN 

C 1. 

r -> 


count 

100 

100 

inn 

ROW 




l 

3 . 119 300 

9.984090 

0. 10 2 2)9 

-> 

0 .099 000 

0.965290 

9.061 109 

% 

3 . 1 72 3 3 o 

9.153700 

9.063609 

4 

3 .0953un 

0. 062 609 

ii, 95 5 79 9 

5 

0.117700 

0. 064 300 

0.062109 

6 

D. 143 50 0 

3. 109 309 

3.059390 

7 

o .161903 

0. 0^7 000 

9.951 499 

3 

0 . 036 BOO 

0.035809 

0.999709 

Q 

0 .093300 

0.035300 

9.063609 

10 

9.102400 

9.039700 

0.066099 

1 1 

0 .086100 

3.959230 

9.052439 

1 2 

0 . 17 4400 

D. 166 700 

9. 9^ 5 3.10 

1 3 

0.030600 

0.073700 

0.064609 

14 

0.072 50-9 

3.063 53 D 

>3.95 6090 

15 

0 . ] 69009 

0. 096600 

0.104209 

16 

0 .114900 

0.089000 

0.958500 

17 

0 . 395900 

3.085690 

0. 087900 

i a 

0 . 200090 

9. 93? 990 

9.965600 

1 O 

9 . 979 330 

0.072000 

0.957303 

2 0 

3 .080 130 

9. 068490 

0.061739 

2 1 

0.073490 

0 . 065 00 0 

0.954599 

22 

0 .1.66 5 99 

3.957532 

9.951699 

23 

0.104300 

9. 089 709 

0.970899 

24 

0.100600 

0.084 300 

9.989499 

2 5 

0 .083300 

0.063000 

0 . 05 8 50 0 

26 

0 . 089 70 0 

0.072600 

9, 06° 30 9 

2 7 

0.075 700 

0.069530 

0 .055 90 9 

23 

0 .094600 

0.937290 

9.061100 

7 9 

0. 097 500 

0.091 190 

0.091090 

30 

0.144400 

3.093 730 

9.059399 

3 1 

0.109300 

0. 965 800 

9.061800 

3 2 

0.082200 

0.077600 

0.964300 

33 

0 . 162 209 

0.063 890 

0. 053839 

34 

0.153200 

0.057299 

0.951809 

3 5 

0.991630 

9.382530 

0.967130 

36 

0 . 128900 

0. 99 7 509 

0.176399 

3 7 

0 .037300 

3.973 700 

0.069900 

33 

9.0931^0 

9 . 08 3 87 0 

9. -0A <5 99 7 

3 9 

9 . 98 7 60 0 

0. 061 209 

9. 95 240 9 

40 

0.117400 

0.147400 

0.084109 

41 

9 .075 600 

0.069000 

0.956600 

42 

0 . 1 09 500 

9.991600 

0. 939397 

43 

3 .038 399 

3. 3°3 299 

0.109800 

44 

0.099100 

9.090 690 

0. 1 07509 

4 5 

9.109990 

9.086700 

0.988700 

46 

0.137600 

9. 064090 

9.96 9730 

47 

0 . 129700 

0. 963 590 

0.954090 

4 a 

0 . 076 80 ^ 

0.073 500 

0.065990 

49 

9 .122 100 

9.208600 

0.073300 

50 

9 . 9°9 590 

0.091 109 

0. 009997 

51 

0.078309 

3.974739 

0 . 364030 

52 

9 . 1 93 009 

9. 193 009 

0. 06-9100 

53 

9 . 116200 

0. 008 on Q 

0. 102790 

54 

3 .107390 

3 . ''8 2 690 

3.98 A 2 0 9 

55 

0 . 1340 00 

9. 165700 

0.994599 


Table 4. - Cost Values for the 100 Monte-Carlo Runs for the 

constant parameter scalar model, using the cautious, 
first order dual and second order dual controllers 
(b(0)=.05, P C0)*1, V=0, c=l, W=.01, No Control Cost=l) 
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CAUTIOUS FIRST ORDER SECOND ORDER 

DUAL DUAL 


56 

0 . 79 373 .'' 

0.134330 

0 . : iqnim 

5 7 

0.11? 300 

0 . Of 7 900 

0 . 0641U0 

5 3 

0. 16 2 40 0 

0. 063 60 0 

0.058500 

5 9 

0 .166700 

■'. 139 400 

0.056809 

60 

0 .083700 

0. 035 700 

0.060100 

6 1 

0 . 036900 

1 . OQl. 200 

0 , 05 3500 

6 7 

0 .034200 

0. 068 700 

0 . 0653)0 

63 

0.125000 

0 , n 71 59 0 

0. OS 2 500 

64 

0.119 50 0 

0. 077700 

0.060700 

66 

0 .193400 

0. 960 900 

0.061700 

66 

0.075700 

0 . 069400 

0. 061900 

67 

0 . 077200 

0.063900 

0.961190 

68 

0.126300 

9 . 072 600 

0. 070500 

50 

0.191100 

0.060300 

0 . 05 5 50 0 

70 

0 .008200 

0.069500 

0,065500 

7 1 

0.087990 

0* "6^ 300 

n # n r >34Hn 

72 

0.112 800 

0 . 722330 

0.115 30 0 

73 

0 .135100 

3* 109 30 0 

0.060700 

74 

0 . 121 300 

0. 103 500 

0.1 12700 

75 

7.110300 

9.091 10 0 

0.972 9 

7 4 

0 .139000 

0. 739400 

0 * 06 3200 

77 

0.034200 

9,064600 

o # ns e Ron 

73 

0 .075:00 0 

a . 067400 

0.059700 

79 

0.120409 

0.092290 

0. 083400 

3 0 

0.073400 

3. 065 700 

0. 35 58 50 

3 1 

9 .085600 

0. 035 119 

0.084800 

«2 

9.076700 

0.073100 

0,064700 

33 

0.111609 

0 . 087500 

0.062 80'' 

84 

0.103400 

0. OQl 900 

0.070400 

35 

0.095 40 0 

0. 0902O0 

0.097900 

86 

a .100700 

0.077800 

0 . 0 58 60 0 

3 7 

0.076400 

0-. 070 100 

0.066199 

3 a 

0.0877)0 

9. 071000 

0.067900 

8 9 

0 .075300 

3. 067503 

0 . 95 460 0 

90 

0. 135000 

0 . 073 700 

0.053200 

9 1 

0 . 979490 

0 . 96 5 090 

9 . 9 5 *>. 40 3 

92 

0.118700 

0. 064300 

0.061300 

93 

0.103 700 

0. 124090 

0. 04 190 0 

9 4 

a. 09 6 50 9 

0.109100 

u , 06 3 30 0 

95 

0.1 14600 

0. 067400 

0.958700 

96 

n. 08 1990 

0 . 06 4110 

0.056309 

97 

0 .074400 

0. 064500 

0 . 05 5 90 0 

9 8 

0.133600 

0. 0 0 3 2 0 0 

0.062800 

99 

0.091 530 

9 . 90 7 6,03 

9.05 340 9 

10 0 

0.093300 

0. 064000 

0. 057700 


Average Cautious = .10907 



Average First 

Order Dual = 

.08739 


Average Second 

. Order Dual = 

.06927 


Table 4. (Cont.) 
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CAUTIOUS 

FIRST ORDER 

SECOND ORDER 



DUAL 

DUAL 

COLUMN 

C 1 

r -> 

“ 3 

CTJMT 

100 

100 

10 0 

RIM 




l 

0 .274600 

9. 2>)7 4 Of) 

0.131200 

2 

0.400300 

0. 479800 

0.129 100 

3 

3.451 300 

3.1483)0 

1 , 1 66711 

4 

0 . 191100 

0. 183 100 

0.110700 

5 

0.321000 

9 . 747 300 

0. 146800 

6 

0 .833000 

80 0 

0. 128 39 0 

7 

0 . 503 qoo 

0.333500 

0. 106100 

q 

0 .261 900 

0.185500 

9.136730 

9 

3.737100 

0 . 167100 

0. 19 3730 

1 0 

0 . 354000 

0.327500 

0. 124100 

1 1 

0 .784430 

0 . 171900 

0.145291 

1 2 

9.346700 

0. 3 03 600 

0. 16 3200 

1 3 

0. 239690 

0.145400 

0.119010 

1 4 

0. 4Q 4 600 

0. 107200 

0. 101600 

1 5 

0. 64510 0 

0.183700 

0. 140800 

16 

0.211103 

0.245 309 

0.246600 

1 7 

3 . 4 3 7 4 '3 0 

0. 766900 

0. 11 9590 

l A 

0 . 405 70 0 

0.473400 

0. 131 800 

19 

1 . 1563.) 9 

0.131830 

). 1 1 6.30 0 

if) 

0.290400 

0. 733800 

0. 138 800 

2 1 

9.374700 

0.707300 

0. 106 30’) 

7 7 

0 . 465709 

9 . 364300 

0. 117900 

2 3 

0.240000 

0.204400 

0. 129 800 

24 

0 .732 900 

0.318 1)0 

0. 143 490 

25 

9.259500 

). 21) 700 

0. 141400 

2 6 

0 .3 75 30 O 

0, 988200 

0.115700 

2 7 

0. 155 699 

9.117409 

0. 105330 

? q 

0 ,lf»7600 

0. 153000 

0.119600 

79 

0 . 194700 

0. 163200 

0.131700 

3 0 

0.758700 

0.257900 

0, 120 300 

31 

9.350100 

0. 343 700 

0. 130500 

3 2 

0.203200 

0. 18 4503 

0.174190 

3 3 

0.557400 

0. 766800 

0. 108700 

? 4 

0. 377900 

0. 293 000 

0.420400 

3 5 

0.77 OR 00 

3 . 344400 

0. 11 7 800 

3 6 

9 . 517600 

0.331 903 

0. 143000 

37 

0.330200 

•1.193 09 9 

9. 13213'* 

38 

0 . 704 q 00 

3. 865 300 

0 .14 5 ID O 

39 

0. 310800 

0.215800 

0.109800 

49 

0.28 8500 

1.197900 

0. 17 9130 

4 1 

0 . 1.51 30 0 

0. 119 200 

0. 105600 

4 2 

0.340600 

0.216700 

0.144300 

43 

0 .329500 

0.269400 

0. 140400 

4 4 

0.176600 

0. 123 800 

0. 120700 

4 5 

0.289700 

0. 193600 

0. 119690 

4 6 

0.396500 

3. 3 45190 

0 . 705000 

47 

0.373700 

0. 175000 

0.137600 

4 3 

0.29.3 -59.0 

0. 190 701 

9 .12 63 3 3 

4 9 

0.353500 

0. 187 300 

0.219400 

5 0 

0 . 213 700 

9.747 093 

0.386000 

6 1 

0 .43 7 30 0 

3.197100 

. 0.148499 

5 7 

0.305000 

O. 206700 

0. 140000 

53 

0.48 0000 

9.4 98 800 

0 . 13 83 90 

54 

9.301890 

9. 743 830 

0 .11? 7)0 


Table 5. - Cost Values for the 1Q.Q Monte-Carlo Runs for the 

constant parameter scalar model, using the cautious, 
first order dual and second order dual controllers 
(b(0) = . 05, P b (0) = l, V=0, c-1, W=. 1, No Control Cost=l) 
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Table 5. 


oi* mm 




CAUTIOUS 

FIRST ORDER 

SECOND ORDER 



DUAL 

DUAL 

55 

0. ISO 100 

0. 167200 

0. 15 1300 

56 

0 . 56 7909 

3.411 600 

0.- 11 480 r 

5 7 

0 .290100 

n. 1.07 ooo 

0.124100 

5 5 

0.44239Q 

0.775290 

0. 154700 

*5 9 

0.273500 

0 .786390 

n. 1 1 5 loo 

60 

9 .481900 

0. 153 300 

o. 1 An son 

6 1 

0.3095 0 0 

144 8 jn 

1.124290 

62 

0 . 363690 

3. 201 10 0 

1. 120800 

53 

0. 352700 

0. 3.7 7 200 

0. 12 7500 

64 

0.333400 

0.323 1)0 

0 ,2163 3 O 

65 

0.330700 

0. 223000 

0. 142000 

56 

0.347800 

0.203600 

0.705200 

67 

0 .403700 

D. 22 5 8.00 

0 . 124200 

63 

0 .287000 
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i Measurement 

; Noise Covariance 

W 

! 

Average Cost 

Cautious 

First Order 
Dual 

Second Order 
Dual 

| .01 

.475 

.469 

.458 

1 

i .1 

1 

.623 

• 

ON 

o 

CO 

.514 

1 


Table 6. Average Cost for the three controllers on the time 

varying parameter model (b(0) = .05 , P^(0)=1 } V-.l, c=l ) 


— ~ 

' 

Measurement 
Noise Covariance 
v? 

i 

Average Cost 

Cautious 

First Order 
Dual 

Second Order 
Dual 

.01 

.109 

.087 

.069 

i .i 

i 

.359 

.250 

•142 


Table 7. Average Cost for three controllers on the Constant 
Parameter Model (b(0)=.05, P^(0)=1 , V=0, c=l ) 
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